[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:43:23 dist-test-slave-1jjb krb5kdc[13231](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:43:23 dist-test-slave-1jjb krb5kdc[13231](info): set up 2 sockets
Aug 09 04:43:23 dist-test-slave-1jjb krb5kdc[13231](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 04:43:24.571372 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.003s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:43:24 dist-test-slave-1jjb krb5kdc[13231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:43:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:43:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:43:24.772296 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:44523
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:43581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:44523
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:25.191668 13245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:25.192138 13245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:25.192263 13245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:25.199900 13245 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:43:25.200068 13245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:25.200278 13245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:25.200353 13245 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:43:25.200399 13245 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:43:25.211429 13245 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:43581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:44523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:44523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:25.213318 13245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:25.215075 13245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:25.234561 13250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:25.235050 13253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:25.235195 13251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:25.236215 13245 server_base.cc:1034] running on GCE node
I20260809 04:43:25.237262 13245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:25.239353 13245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:25.240576 13245 hybrid_clock.cc:648] HybridClock initialized: now 1786250605240463 us; error 122 us; skew 500 ppm
I20260809 04:43:25.241091 13245 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:25 dist-test-slave-1jjb krb5kdc[13231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250605, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:25.244526 13245 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:43:25.246564 13245 webserver.cc:460] Webserver started at http://127.12.234.62:37943/ using document root <none> and password file <none>
I20260809 04:43:25.247534 13245 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:25.247630 13245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:25.247999 13245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:25.252156 13245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "fb646f595cfb4a2d9163eb1ef6b9dddb"
format_stamp: "Formatted at 2026-08-09 04:43:25 on dist-test-slave-1jjb"
server_key: "a6c9274ef25867051a1ab7d75aae7f26"
server_key_iv: "59a821e75daffc51e7077e628beba20d"
server_key_version: "encryptionkey@0"
I20260809 04:43:25.253022 13245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "fb646f595cfb4a2d9163eb1ef6b9dddb"
format_stamp: "Formatted at 2026-08-09 04:43:25 on dist-test-slave-1jjb"
server_key: "a6c9274ef25867051a1ab7d75aae7f26"
server_key_iv: "59a821e75daffc51e7077e628beba20d"
server_key_version: "encryptionkey@0"
I20260809 04:43:25.260582 13245 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260809 04:43:25.265992 13260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:25.267742 13245 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260809 04:43:25.267979 13245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "fb646f595cfb4a2d9163eb1ef6b9dddb"
format_stamp: "Formatted at 2026-08-09 04:43:25 on dist-test-slave-1jjb"
server_key: "a6c9274ef25867051a1ab7d75aae7f26"
server_key_iv: "59a821e75daffc51e7077e628beba20d"
server_key_version: "encryptionkey@0"
I20260809 04:43:25.268275 13245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:25.292163 13245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:25.296615 13245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:25.296936 13245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:25.320489 13321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:44523 every 8 connection(s)
I20260809 04:43:25.320536 13245 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:44523
I20260809 04:43:25.322700 13245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:43:25.324725 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 13245
I20260809 04:43:25.325057 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:43:25.327032 13224 external_mini_cluster.cc:1428] Setting key 8ce30d64d8724d2f30309dfd7084550c
I20260809 04:43:25.328786 13322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:25.344770 13322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb646f595cfb4a2d9163eb1ef6b9dddb: Bootstrap starting.
I20260809 04:43:25.349133 13322 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb646f595cfb4a2d9163eb1ef6b9dddb: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:25.350592 13322 log.cc:824] T 00000000000000000000000000000000 P fb646f595cfb4a2d9163eb1ef6b9dddb: Log is configured to *not* fsync() on all Append() calls
I20260809 04:43:25.354835 13322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb646f595cfb4a2d9163eb1ef6b9dddb: No bootstrap required, opened a new log
Aug 09 04:43:25 dist-test-slave-1jjb krb5kdc[13231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:25.363772 13322 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb646f595cfb4a2d9163eb1ef6b9dddb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb646f595cfb4a2d9163eb1ef6b9dddb" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44523 } }
I20260809 04:43:25.364209 13322 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb646f595cfb4a2d9163eb1ef6b9dddb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:25.364403 13322 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb646f595cfb4a2d9163eb1ef6b9dddb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb646f595cfb4a2d9163eb1ef6b9dddb, State: Initialized, Role: FOLLOWER
I20260809 04:43:25.365576 13322 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb646f595cfb4a2d9163eb1ef6b9dddb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb646f595cfb4a2d9163eb1ef6b9dddb" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44523 } }
I20260809 04:43:25.365923 13322 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb646f595cfb4a2d9163eb1ef6b9dddb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:43:25.366199 13322 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb646f595cfb4a2d9163eb1ef6b9dddb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:43:25.366374 13322 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb646f595cfb4a2d9163eb1ef6b9dddb [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:25.367654 13325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:25.334330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52160 (local address 127.12.234.62:44523)
0809 04:43:25.335414 (+ 1084us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:25.335432 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:25.335486 (+ 54us) server_negotiation.cc:407] Connection header received
0809 04:43:25.343420 (+ 7934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:25.343439 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:25.343905 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:25.344291 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:25.347046 (+ 2755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:25.348483 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:25.350176 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:25.351029 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:25.351324 (+ 295us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:25.358554 (+ 7230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:25.358605 (+ 51us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:25.358632 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:25.358664 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:25.361654 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:25.362428 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:25.362436 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:25.362441 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:25.362669 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:25.363137 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:25.363143 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:25.363146 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:25.363694 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:25.364097 (+ 403us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:25.365432 (+ 1335us) server_negotiation.cc:299] Negotiation successful
0809 04:43:25.365877 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":361,"threads_started":1}
I20260809 04:43:25.369175 13322 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb646f595cfb4a2d9163eb1ef6b9dddb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb646f595cfb4a2d9163eb1ef6b9dddb" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44523 } }
I20260809 04:43:25.370080 13322 leader_election.cc:302] T 00000000000000000000000000000000 P fb646f595cfb4a2d9163eb1ef6b9dddb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb646f595cfb4a2d9163eb1ef6b9dddb; no voters:
I20260809 04:43:25.370925 13322 leader_election.cc:288] T 00000000000000000000000000000000 P fb646f595cfb4a2d9163eb1ef6b9dddb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:43:25.371268 13327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb646f595cfb4a2d9163eb1ef6b9dddb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:43:25.372579 13327 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb646f595cfb4a2d9163eb1ef6b9dddb [term 1 LEADER]: Becoming Leader. State: Replica: fb646f595cfb4a2d9163eb1ef6b9dddb, State: Running, Role: LEADER
I20260809 04:43:25.373664 13327 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb646f595cfb4a2d9163eb1ef6b9dddb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb646f595cfb4a2d9163eb1ef6b9dddb" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44523 } }
I20260809 04:43:25.374234 13322 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb646f595cfb4a2d9163eb1ef6b9dddb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:43:25.379535 13328 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb646f595cfb4a2d9163eb1ef6b9dddb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb646f595cfb4a2d9163eb1ef6b9dddb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb646f595cfb4a2d9163eb1ef6b9dddb" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44523 } } }
I20260809 04:43:25.379935 13328 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb646f595cfb4a2d9163eb1ef6b9dddb [sys.catalog]: This master's current role is: LEADER
I20260809 04:43:25.380546 13329 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb646f595cfb4a2d9163eb1ef6b9dddb [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb646f595cfb4a2d9163eb1ef6b9dddb. Latest consensus state: current_term: 1 leader_uuid: "fb646f595cfb4a2d9163eb1ef6b9dddb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb646f595cfb4a2d9163eb1ef6b9dddb" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44523 } } }
I20260809 04:43:25.380833 13329 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb646f595cfb4a2d9163eb1ef6b9dddb [sys.catalog]: This master's current role is: LEADER
I20260809 04:43:25.385283 13337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:43:25.392024 13337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:43:25.405438 13337 catalog_manager.cc:1347] Generated new cluster ID: 4fce961d2bb74af28275865b929675f3
I20260809 04:43:25.405545 13337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:43:25.428791 13337 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:43:25.430107 13337 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:43:25.444890 13337 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb646f595cfb4a2d9163eb1ef6b9dddb: Generated new TSK 0
I20260809 04:43:25.445850 13337 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
I20260809 04:43:25.492646 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:0
--local_ip_for_outbound_sockets=127.12.234.1
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44523
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:43581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:25.820750 13349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:25.821050 13349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:25.821230 13349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:25.828730 13349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:25.828856 13349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:25.828976 13349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:43:25.839039 13349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:43581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44523
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:25.840344 13349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:25.841647 13349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:25.855439 13355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:25.855489 13356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:25.855502 13358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:25.856802 13349 server_base.cc:1034] running on GCE node
I20260809 04:43:25.857381 13349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:25.858286 13349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:25.859529 13349 hybrid_clock.cc:648] HybridClock initialized: now 1786250605859463 us; error 60 us; skew 500 ppm
I20260809 04:43:25.859872 13349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:25 dist-test-slave-1jjb krb5kdc[13231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250605, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:25.863061 13349 init.cc:377] Logged in from keytab as kudu/127.12.234.1@KRBTEST.COM (short username kudu)
I20260809 04:43:25.864835 13349 webserver.cc:460] Webserver started at http://127.12.234.1:35253/ using document root <none> and password file <none>
I20260809 04:43:25.865548 13349 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:25.865696 13349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:25.866029 13349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:25.869086 13349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-0/data/instance:
uuid: "2fe48828ad8745aba13a41a0588bd21d"
format_stamp: "Formatted at 2026-08-09 04:43:25 on dist-test-slave-1jjb"
server_key: "d56127aee4a3898d0f5f3e002045faea"
server_key_iv: "90b94b17193440b2f2a832d621848363"
server_key_version: "encryptionkey@0"
I20260809 04:43:25.869913 13349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance:
uuid: "2fe48828ad8745aba13a41a0588bd21d"
format_stamp: "Formatted at 2026-08-09 04:43:25 on dist-test-slave-1jjb"
server_key: "d56127aee4a3898d0f5f3e002045faea"
server_key_iv: "90b94b17193440b2f2a832d621848363"
server_key_version: "encryptionkey@0"
I20260809 04:43:25.875382 13349 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 04:43:25.878974 13366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:25.880426 13349 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:43:25.880599 13349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "2fe48828ad8745aba13a41a0588bd21d"
format_stamp: "Formatted at 2026-08-09 04:43:25 on dist-test-slave-1jjb"
server_key: "d56127aee4a3898d0f5f3e002045faea"
server_key_iv: "90b94b17193440b2f2a832d621848363"
server_key_version: "encryptionkey@0"
I20260809 04:43:25.880798 13349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:25.905053 13349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:25.909384 13349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:25.909691 13349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:25.910828 13349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:43:25.912521 13349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:43:25.912631 13349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:25.912842 13349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:43:25.912932 13349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:25.951474 13349 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:34069
I20260809 04:43:25.951565 13488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:34069 every 8 connection(s)
I20260809 04:43:25.952879 13349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:43:25.958659 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 13349
I20260809 04:43:25.958923 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance
I20260809 04:43:25.959569 13224 external_mini_cluster.cc:1428] Setting key ff4b0d84ce89a3a72575142a0a6fd0c0
Aug 09 04:43:25 dist-test-slave-1jjb krb5kdc[13231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250605, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:25.975504 13491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:25.956407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:37341 (local address 127.12.234.62:44523)
0809 04:43:25.956894 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:25.956899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:25.957933 (+ 1034us) server_negotiation.cc:407] Connection header received
0809 04:43:25.959980 (+ 2047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:25.959987 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:25.960100 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:25.960237 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:25.961916 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:25.963004 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:25.964647 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:25.965471 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:25.965615 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:25.969502 (+ 3887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:25.969507 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:25.969510 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:25.969542 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:25.972161 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:25.972968 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:25.972976 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:25.972985 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:25.973109 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:25.973561 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:25.973572 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:25.973574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:25.973906 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:25.974086 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:25.974905 (+ 819us) server_negotiation.cc:299] Negotiation successful
0809 04:43:25.975140 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":264,"threads_started":1}
I20260809 04:43:25.976637 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:25.957031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44523 (local address 127.12.234.1:37341)
0809 04:43:25.957768 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:25.957812 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:25.959681 (+ 1869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:25.960482 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:25.960494 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:25.961066 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:25.961741 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:25.961773 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:25.963146 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:25.963150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:25.964493 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:25.964505 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:25.965690 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:25.965695 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:25.965894 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:25.966616 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:25.966639 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:25.969264 (+ 2625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:25.972295 (+ 3031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:25.972305 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:25.972323 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:25.972842 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:25.973226 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:25.973231 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:25.973235 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:25.973433 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:25.974038 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:25.974046 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:25.974426 (+ 380us) client_negotiation.cc:769] Sending connection context
0809 04:43:25.974788 (+ 362us) client_negotiation.cc:240] Negotiation successful
0809 04:43:25.975311 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":317,"threads_started":1}
I20260809 04:43:25.978315 13489 heartbeater.cc:347] Connected to a master server at 127.12.234.62:44523
I20260809 04:43:25.978834 13489 heartbeater.cc:464] Registering TS with master...
I20260809 04:43:25.979929 13489 heartbeater.cc:511] Master 127.12.234.62:44523 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.12.234.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.2@KRBTEST.COM" created.
I20260809 04:43:25.984141 13281 ts_manager.cc:194] Registered new tserver with Master: 2fe48828ad8745aba13a41a0588bd21d (127.12.234.1:34069)
I20260809 04:43:25.986465 13281 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.1@KRBTEST.COM'} at 127.12.234.1:37341
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
I20260809 04:43:26.009328 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:0
--local_ip_for_outbound_sockets=127.12.234.2
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44523
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:43581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:26.357682 13495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:26.357903 13495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:26.357982 13495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:26.364787 13495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:26.364871 13495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:26.364977 13495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:43:26.374420 13495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:43581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44523
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:26.375526 13495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:26.376773 13495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:26.390055 13500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:26.390070 13502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:26.390167 13504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:26.391022 13495 server_base.cc:1034] running on GCE node
I20260809 04:43:26.391474 13495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:26.392292 13495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:26.393487 13495 hybrid_clock.cc:648] HybridClock initialized: now 1786250606393458 us; error 29 us; skew 500 ppm
I20260809 04:43:26.393770 13495 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:26 dist-test-slave-1jjb krb5kdc[13231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250606, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:26.396471 13495 init.cc:377] Logged in from keytab as kudu/127.12.234.2@KRBTEST.COM (short username kudu)
I20260809 04:43:26.397665 13495 webserver.cc:460] Webserver started at http://127.12.234.2:33053/ using document root <none> and password file <none>
I20260809 04:43:26.398238 13495 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:26.398370 13495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:26.398640 13495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:26.401222 13495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-1/data/instance:
uuid: "0f6c8962d26d4902a8cf96389453d3c3"
format_stamp: "Formatted at 2026-08-09 04:43:26 on dist-test-slave-1jjb"
server_key: "2ea3f4a8d9f86d15db048ffe7dbc430b"
server_key_iv: "1f7f2579fde91cb6853cd4b01ca2ed28"
server_key_version: "encryptionkey@0"
I20260809 04:43:26.401954 13495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance:
uuid: "0f6c8962d26d4902a8cf96389453d3c3"
format_stamp: "Formatted at 2026-08-09 04:43:26 on dist-test-slave-1jjb"
server_key: "2ea3f4a8d9f86d15db048ffe7dbc430b"
server_key_iv: "1f7f2579fde91cb6853cd4b01ca2ed28"
server_key_version: "encryptionkey@0"
I20260809 04:43:26.406589 13495 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260809 04:43:26.409725 13512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:26.410831 13495 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:43:26.410987 13495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "0f6c8962d26d4902a8cf96389453d3c3"
format_stamp: "Formatted at 2026-08-09 04:43:26 on dist-test-slave-1jjb"
server_key: "2ea3f4a8d9f86d15db048ffe7dbc430b"
server_key_iv: "1f7f2579fde91cb6853cd4b01ca2ed28"
server_key_version: "encryptionkey@0"
I20260809 04:43:26.411151 13495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:26.428532 13495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:26.432312 13495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:26.432585 13495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:26.433465 13495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:43:26.434584 13495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:43:26.434679 13495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:26.434808 13495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:43:26.434878 13495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:26.484823 13495 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:40297
I20260809 04:43:26.484884 13635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:40297 every 8 connection(s)
I20260809 04:43:26.486632 13495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:43:26.489037 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 13495
I20260809 04:43:26.489315 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance
I20260809 04:43:26.489884 13224 external_mini_cluster.cc:1428] Setting key 0489de82f3d2473ff12ea5d457966921
Aug 09 04:43:26 dist-test-slave-1jjb krb5kdc[13231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250606, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.234.3@KRBTEST.COM; defaulting to no policy
I20260809 04:43:26.517056 13638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:26.490594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:43901 (local address 127.12.234.62:44523)
0809 04:43:26.494099 (+ 3505us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:26.494107 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:26.494135 (+ 28us) server_negotiation.cc:407] Connection header received
0809 04:43:26.497668 (+ 3533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:26.497675 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:26.497763 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:26.497871 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:26.500441 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:26.501521 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:26.503395 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:26.504326 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:26.504476 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:26.508926 (+ 4450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:26.508933 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:26.508938 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:26.508980 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:26.513097 (+ 4117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:26.514008 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:26.514013 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:26.514017 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:26.514157 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:26.514646 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:26.514651 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:26.514655 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:26.515055 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:26.515242 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:26.516228 (+ 986us) server_negotiation.cc:299] Negotiation successful
0809 04:43:26.516589 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3369,"thread_start_us":3277,"threads_started":1}
I20260809 04:43:26.517529 13639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:26.491340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44523 (local address 127.12.234.2:43901)
0809 04:43:26.492808 (+ 1468us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:26.492893 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:26.497227 (+ 4334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:26.498318 (+ 1091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:26.498331 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:26.499287 (+ 956us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:26.500172 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:26.500210 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:26.501654 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:26.501658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:26.503202 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:26.503218 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:26.504571 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:26.504581 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:26.504926 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:26.505816 (+ 890us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:26.505871 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:26.508664 (+ 2793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:26.513259 (+ 4595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:26.513268 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:26.513282 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:26.513884 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:26.514283 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:26.514287 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:26.514291 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:26.514529 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:26.515188 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:26.515195 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:26.515642 (+ 447us) client_negotiation.cc:769] Sending connection context
0809 04:43:26.515955 (+ 313us) client_negotiation.cc:240] Negotiation successful
0809 04:43:26.516505 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1070,"thread_start_us":1718,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.3@KRBTEST.COM" created.
I20260809 04:43:26.518949 13636 heartbeater.cc:347] Connected to a master server at 127.12.234.62:44523
I20260809 04:43:26.519219 13636 heartbeater.cc:464] Registering TS with master...
I20260809 04:43:26.519960 13636 heartbeater.cc:511] Master 127.12.234.62:44523 requested a full tablet report, sending...
I20260809 04:43:26.521492 13281 ts_manager.cc:194] Registered new tserver with Master: 0f6c8962d26d4902a8cf96389453d3c3 (127.12.234.2:40297)
I20260809 04:43:26.522429 13281 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.2@KRBTEST.COM'} at 127.12.234.2:43901
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
I20260809 04:43:26.539320 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:0
--local_ip_for_outbound_sockets=127.12.234.3
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44523
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:43581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:26.918350 13642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:26.918655 13642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:26.918754 13642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:26.926608 13642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:26.926723 13642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:26.926839 13642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:43:26.937608 13642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:43581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44523
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:26.939036 13642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:26.940806 13642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:26.955312 13647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:26.955282 13648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:26.955349 13650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:26.958746 13642 server_base.cc:1034] running on GCE node
I20260809 04:43:26.959466 13642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:26.960508 13642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:26.961745 13642 hybrid_clock.cc:648] HybridClock initialized: now 1786250606961700 us; error 55 us; skew 500 ppm
I20260809 04:43:26.962081 13642 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:26 dist-test-slave-1jjb krb5kdc[13231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250606, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:26.965605 13642 init.cc:377] Logged in from keytab as kudu/127.12.234.3@KRBTEST.COM (short username kudu)
I20260809 04:43:26.967252 13642 webserver.cc:460] Webserver started at http://127.12.234.3:36993/ using document root <none> and password file <none>
I20260809 04:43:26.968035 13642 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:26.968254 13642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:26.968600 13642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:26.971773 13642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-2/data/instance:
uuid: "8a0f20527bdc4c9ba02e36fad7d35716"
format_stamp: "Formatted at 2026-08-09 04:43:26 on dist-test-slave-1jjb"
server_key: "5dfb821d0245d0cf859c00b8d24e6eae"
server_key_iv: "2c5701805bbed4b115a8b031170153f5"
server_key_version: "encryptionkey@0"
I20260809 04:43:26.972739 13642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance:
uuid: "8a0f20527bdc4c9ba02e36fad7d35716"
format_stamp: "Formatted at 2026-08-09 04:43:26 on dist-test-slave-1jjb"
server_key: "5dfb821d0245d0cf859c00b8d24e6eae"
server_key_iv: "2c5701805bbed4b115a8b031170153f5"
server_key_version: "encryptionkey@0"
I20260809 04:43:26.978339 13642 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260809 04:43:26.982013 13657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:26.983527 13642 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:43:26.983753 13642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "8a0f20527bdc4c9ba02e36fad7d35716"
format_stamp: "Formatted at 2026-08-09 04:43:26 on dist-test-slave-1jjb"
server_key: "5dfb821d0245d0cf859c00b8d24e6eae"
server_key_iv: "2c5701805bbed4b115a8b031170153f5"
server_key_version: "encryptionkey@0"
I20260809 04:43:26.983969 13642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:26.990156 13489 heartbeater.cc:503] Master 127.12.234.62:44523 was elected leader, sending a full tablet report...
I20260809 04:43:27.004926 13642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:27.009588 13642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:27.009838 13642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:27.010874 13642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:43:27.012410 13642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:43:27.012492 13642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:27.012586 13642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:43:27.012632 13642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:27.053166 13642 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:38205
I20260809 04:43:27.053412 13778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:38205 every 8 connection(s)
I20260809 04:43:27.054536 13642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:43:27.057197 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 13642
I20260809 04:43:27.057441 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance
I20260809 04:43:27.058037 13224 external_mini_cluster.cc:1428] Setting key 77d1a837286ffae5afb62a92f8644484
Aug 09 04:43:27 dist-test-slave-1jjb krb5kdc[13231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250606, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:27.077713 13781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:27.057509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:34285 (local address 127.12.234.62:44523)
0809 04:43:27.058019 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:27.058023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:27.059505 (+ 1482us) server_negotiation.cc:407] Connection header received
0809 04:43:27.062312 (+ 2807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:27.062322 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:27.062389 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:27.062477 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:27.064218 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:27.065013 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:27.066680 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:27.067128 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:27.067225 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:27.070756 (+ 3531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:27.070761 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:27.070764 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:27.070797 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:27.073985 (+ 3188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:27.074965 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:27.074981 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:27.074984 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:27.075152 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:27.075778 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:27.075787 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:27.075790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:27.076120 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:27.076303 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:27.077054 (+ 751us) server_negotiation.cc:299] Negotiation successful
0809 04:43:27.077284 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":296,"threads_started":1}
I20260809 04:43:27.078722 13782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:27.058268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44523 (local address 127.12.234.3:34285)
0809 04:43:27.059317 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:27.059370 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:27.061968 (+ 2598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:27.062823 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:27.062836 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:27.063410 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:27.064004 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:27.064022 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:27.065144 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:27.065147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:27.066548 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:27.066561 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:27.067323 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:27.067328 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:27.067524 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:27.068233 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:27.068261 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:27.070551 (+ 2290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:27.074134 (+ 3583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:27.074143 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:27.074163 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:27.074824 (+ 661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:27.075334 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:27.075342 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:27.075345 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:27.075643 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:27.076270 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:27.076278 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:27.076671 (+ 393us) client_negotiation.cc:769] Sending connection context
0809 04:43:27.076971 (+ 300us) client_negotiation.cc:240] Negotiation successful
0809 04:43:27.077557 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"thread_start_us":477,"threads_started":1}
I20260809 04:43:27.080271 13779 heartbeater.cc:347] Connected to a master server at 127.12.234.62:44523
I20260809 04:43:27.080600 13779 heartbeater.cc:464] Registering TS with master...
I20260809 04:43:27.081457 13779 heartbeater.cc:511] Master 127.12.234.62:44523 requested a full tablet report, sending...
I20260809 04:43:27.082985 13281 ts_manager.cc:194] Registered new tserver with Master: 8a0f20527bdc4c9ba02e36fad7d35716 (127.12.234.3:38205)
I20260809 04:43:27.083896 13281 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.3@KRBTEST.COM'} at 127.12.234.3:34285
I20260809 04:43:27.088370 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 04:43:27 dist-test-slave-1jjb krb5kdc[13231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250607, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 04:43:27 dist-test-slave-1jjb krb5kdc[13231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250607, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.234.1@KRBTEST.COM
I20260809 04:43:27.144064 13787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:27.127630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58982 (local address 127.12.234.1:34069)
0809 04:43:27.128449 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:27.128456 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:27.128485 (+ 29us) server_negotiation.cc:407] Connection header received
0809 04:43:27.128560 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:27.128566 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:27.128779 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:27.129044 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:27.130591 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:27.131595 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:27.132959 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:27.133534 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:27.133633 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:27.137994 (+ 4361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:27.138007 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:27.138024 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:27.138065 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:27.141162 (+ 3097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:27.141923 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:27.141931 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:27.141934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:27.142065 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:27.142541 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:27.142546 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:27.142548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:27.142915 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:27.143138 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:27.143498 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 04:43:27.143701 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":520,"threads_started":1}
W20260809 04:43:27.145561 13398 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58982
Aug 09 04:43:27 dist-test-slave-1jjb krb5kdc[13231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 04:43:27 dist-test-slave-1jjb krb5kdc[13231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.1@KRBTEST.COM
I20260809 04:43:27.194950 13787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:27.180621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58990 (local address 127.12.234.1:34069)
0809 04:43:27.180900 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:27.180907 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:27.181192 (+ 285us) server_negotiation.cc:407] Connection header received
0809 04:43:27.181352 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:27.181356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:27.181453 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:27.181543 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:27.183074 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:27.183811 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:27.185339 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:27.185857 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:27.185958 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:27.190177 (+ 4219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:27.190183 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:27.190186 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:27.190222 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:27.192740 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:27.193339 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:27.193343 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:27.193345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:27.193460 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:27.193762 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:27.193765 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:27.193767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:27.194004 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:27.194184 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:27.194428 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 04:43:27.194593 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260809 04:43:27.196969 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 13349
I20260809 04:43:27.210098 13787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:27.197962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59002 (local address 127.12.234.1:34069)
0809 04:43:27.198125 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:27.198128 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:27.198534 (+ 406us) server_negotiation.cc:407] Connection header received
0809 04:43:27.198671 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:27.198674 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:27.198727 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:27.198800 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:27.199920 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:27.200691 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:27.202151 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:27.202574 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:27.202663 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:27.205014 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:27.205019 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:27.205022 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:27.205056 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:27.207665 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:27.208345 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:27.208349 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:27.208351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:27.208490 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:27.208842 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:27.208845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:27.208846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:27.209176 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:27.209353 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:27.209662 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 04:43:27.209815 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 04:43:27.210932 13483 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:43:27.298707 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 13349
I20260809 04:43:27.319527 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 13495
Aug 09 04:43:27 dist-test-slave-1jjb krb5kdc[13231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.2@KRBTEST.COM
I20260809 04:43:27.335614 13794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:27.320482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60292 (local address 127.12.234.2:40297)
0809 04:43:27.321293 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:27.321300 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:27.321326 (+ 26us) server_negotiation.cc:407] Connection header received
0809 04:43:27.321394 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:27.321400 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:27.321605 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:27.321774 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:27.322997 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:27.324005 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:27.325289 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:27.325863 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:27.325999 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:27.330096 (+ 4097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:27.330108 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:27.330124 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:27.330164 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:27.333166 (+ 3002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:27.333739 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:27.333746 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:27.333749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:27.333870 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:27.334193 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:27.334196 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:27.334198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:27.334526 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:27.334737 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:27.335002 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 04:43:27.335196 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":495,"threads_started":1}
I20260809 04:43:27.337007 13631 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:43:27.420730 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 13495
I20260809 04:43:27.441982 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 13642
Aug 09 04:43:27 dist-test-slave-1jjb krb5kdc[13231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.3@KRBTEST.COM
I20260809 04:43:27.457316 13796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:27.442933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53718 (local address 127.12.234.3:38205)
0809 04:43:27.443577 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:27.443584 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:27.443614 (+ 30us) server_negotiation.cc:407] Connection header received
0809 04:43:27.443676 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:27.443683 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:27.443896 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:27.444034 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:27.445330 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:27.446344 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:27.447529 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:27.448054 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:27.448207 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:27.452319 (+ 4112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:27.452331 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:27.452347 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:27.452377 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:27.454889 (+ 2512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:27.455444 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:27.455450 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:27.455452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:27.455556 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:27.455839 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:27.455842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:27.455844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:27.456339 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:27.456600 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:27.456828 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 04:43:27.457024 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":314,"threads_started":1}
I20260809 04:43:27.458704 13774 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:43:27.540915 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 13642
I20260809 04:43:27.562323 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 13245
I20260809 04:43:27.563603 13317 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:43:27.655615 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 13245
2026-08-09T04:43:27Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4210 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:43:27 dist-test-slave-1jjb krb5kdc[13801](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:43:27 dist-test-slave-1jjb krb5kdc[13801](info): set up 2 sockets
Aug 09 04:43:27 dist-test-slave-1jjb krb5kdc[13801](info): commencing operation
krb5kdc: starting...
W20260809 04:43:28.751747 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.007s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:43:28 dist-test-slave-1jjb krb5kdc[13801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:43:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:43:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:43:28.952020 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:42899
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:37663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:42899
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:29.281446 13814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:29.281751 13814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:29.281841 13814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:29.289098 13814 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:43:29.289219 13814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:29.289265 13814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:29.289317 13814 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:43:29.289362 13814 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:43:29.299302 13814 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:37663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:42899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:42899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:29.300645 13814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:29.301937 13814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:29.314201 13820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:29.314244 13821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:29.314181 13824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:29.315120 13814 server_base.cc:1034] running on GCE node
I20260809 04:43:29.315977 13814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:29.317621 13814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:29.318883 13814 hybrid_clock.cc:648] HybridClock initialized: now 1786250609318822 us; error 64 us; skew 500 ppm
I20260809 04:43:29.319303 13814 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:29 dist-test-slave-1jjb krb5kdc[13801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250609, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:29.327252 13814 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:43:29.329677 13814 webserver.cc:460] Webserver started at http://127.12.234.62:45311/ using document root <none> and password file <none>
I20260809 04:43:29.330677 13814 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:29.330839 13814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:29.331187 13814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:29.334477 13814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "ef9d60fdd074407f930eadb8a21ccbfb"
format_stamp: "Formatted at 2026-08-09 04:43:29 on dist-test-slave-1jjb"
server_key: "d6638f3f57427af97fcebd33b9485ecd"
server_key_iv: "889f3481b7737a4b7fa581fdd720d01d"
server_key_version: "encryptionkey@0"
I20260809 04:43:29.335253 13814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "ef9d60fdd074407f930eadb8a21ccbfb"
format_stamp: "Formatted at 2026-08-09 04:43:29 on dist-test-slave-1jjb"
server_key: "d6638f3f57427af97fcebd33b9485ecd"
server_key_iv: "889f3481b7737a4b7fa581fdd720d01d"
server_key_version: "encryptionkey@0"
I20260809 04:43:29.342136 13814 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260809 04:43:29.346066 13831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:29.347510 13814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 04:43:29.347718 13814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "ef9d60fdd074407f930eadb8a21ccbfb"
format_stamp: "Formatted at 2026-08-09 04:43:29 on dist-test-slave-1jjb"
server_key: "d6638f3f57427af97fcebd33b9485ecd"
server_key_iv: "889f3481b7737a4b7fa581fdd720d01d"
server_key_version: "encryptionkey@0"
I20260809 04:43:29.347944 13814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:29.378721 13814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:29.383303 13814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:29.383620 13814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:29.403148 13814 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:42899
I20260809 04:43:29.403173 13885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:42899 every 8 connection(s)
I20260809 04:43:29.404860 13814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:43:29.409274 13886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:29.412758 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 13814
I20260809 04:43:29.413031 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:43:29.413872 13224 external_mini_cluster.cc:1428] Setting key fc49a5157d6850d355e49719936274e7
I20260809 04:43:29.423166 13886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef9d60fdd074407f930eadb8a21ccbfb: Bootstrap starting.
I20260809 04:43:29.427109 13886 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef9d60fdd074407f930eadb8a21ccbfb: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:29.428505 13886 log.cc:824] T 00000000000000000000000000000000 P ef9d60fdd074407f930eadb8a21ccbfb: Log is configured to *not* fsync() on all Append() calls
Aug 09 04:43:29 dist-test-slave-1jjb krb5kdc[13801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:29.432101 13886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef9d60fdd074407f930eadb8a21ccbfb: No bootstrap required, opened a new log
I20260809 04:43:29.440412 13889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:29.416171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40842 (local address 127.12.234.62:42899)
0809 04:43:29.417142 (+ 971us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:29.417154 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:29.417196 (+ 42us) server_negotiation.cc:407] Connection header received
0809 04:43:29.418856 (+ 1660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:29.418882 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:29.419312 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:29.419670 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:29.421562 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:29.422880 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:29.424609 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:29.425304 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:29.425438 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:29.430351 (+ 4913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:29.430361 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:29.430380 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:29.430426 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:29.435038 (+ 4612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:29.435832 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:29.435842 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:29.435851 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:29.436323 (+ 472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:29.436984 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:29.436995 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:29.436999 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:29.437915 (+ 916us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:29.438235 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:29.438792 (+ 557us) server_negotiation.cc:299] Negotiation successful
0809 04:43:29.439255 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":755,"thread_start_us":538,"threads_started":1}
I20260809 04:43:29.442826 13886 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef9d60fdd074407f930eadb8a21ccbfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9d60fdd074407f930eadb8a21ccbfb" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42899 } }
I20260809 04:43:29.443277 13886 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef9d60fdd074407f930eadb8a21ccbfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:29.443408 13886 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef9d60fdd074407f930eadb8a21ccbfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef9d60fdd074407f930eadb8a21ccbfb, State: Initialized, Role: FOLLOWER
I20260809 04:43:29.444051 13886 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef9d60fdd074407f930eadb8a21ccbfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9d60fdd074407f930eadb8a21ccbfb" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42899 } }
I20260809 04:43:29.444352 13886 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef9d60fdd074407f930eadb8a21ccbfb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:43:29.444526 13886 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef9d60fdd074407f930eadb8a21ccbfb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:43:29.444712 13886 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef9d60fdd074407f930eadb8a21ccbfb [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:29.447312 13886 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef9d60fdd074407f930eadb8a21ccbfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9d60fdd074407f930eadb8a21ccbfb" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42899 } }
I20260809 04:43:29.447849 13886 leader_election.cc:302] T 00000000000000000000000000000000 P ef9d60fdd074407f930eadb8a21ccbfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef9d60fdd074407f930eadb8a21ccbfb; no voters:
I20260809 04:43:29.448509 13886 leader_election.cc:288] T 00000000000000000000000000000000 P ef9d60fdd074407f930eadb8a21ccbfb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:43:29.448660 13891 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef9d60fdd074407f930eadb8a21ccbfb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:43:29.449358 13891 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef9d60fdd074407f930eadb8a21ccbfb [term 1 LEADER]: Becoming Leader. State: Replica: ef9d60fdd074407f930eadb8a21ccbfb, State: Running, Role: LEADER
I20260809 04:43:29.449919 13891 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef9d60fdd074407f930eadb8a21ccbfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9d60fdd074407f930eadb8a21ccbfb" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42899 } }
I20260809 04:43:29.450672 13886 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef9d60fdd074407f930eadb8a21ccbfb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:43:29.453349 13892 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef9d60fdd074407f930eadb8a21ccbfb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef9d60fdd074407f930eadb8a21ccbfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9d60fdd074407f930eadb8a21ccbfb" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42899 } } }
I20260809 04:43:29.453423 13893 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef9d60fdd074407f930eadb8a21ccbfb [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef9d60fdd074407f930eadb8a21ccbfb. Latest consensus state: current_term: 1 leader_uuid: "ef9d60fdd074407f930eadb8a21ccbfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef9d60fdd074407f930eadb8a21ccbfb" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42899 } } }
I20260809 04:43:29.453786 13892 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef9d60fdd074407f930eadb8a21ccbfb [sys.catalog]: This master's current role is: LEADER
I20260809 04:43:29.453856 13893 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef9d60fdd074407f930eadb8a21ccbfb [sys.catalog]: This master's current role is: LEADER
I20260809 04:43:29.454707 13897 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:43:29.461617 13897 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:43:29.474351 13897 catalog_manager.cc:1347] Generated new cluster ID: 9970e87d9fd84f6ab4db921768688263
I20260809 04:43:29.474514 13897 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:43:29.490545 13897 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:43:29.492430 13897 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:43:29.506242 13897 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef9d60fdd074407f930eadb8a21ccbfb: Generated new TSK 0
I20260809 04:43:29.507243 13897 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
I20260809 04:43:29.564383 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:0
--local_ip_for_outbound_sockets=127.12.234.1
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:42899
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:37663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:29.902711 13912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:29.902979 13912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:29.903131 13912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:29.910251 13912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:29.910357 13912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:29.910570 13912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:43:29.920238 13912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:37663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:42899
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:29.921496 13912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:29.922909 13912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:29.936715 13920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:29.936779 13917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:29.936921 13918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:29.937575 13912 server_base.cc:1034] running on GCE node
I20260809 04:43:29.938289 13912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:29.939349 13912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:29.940577 13912 hybrid_clock.cc:648] HybridClock initialized: now 1786250609940517 us; error 63 us; skew 500 ppm
I20260809 04:43:29.940984 13912 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:29 dist-test-slave-1jjb krb5kdc[13801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250609, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:29.944216 13912 init.cc:377] Logged in from keytab as kudu/127.12.234.1@KRBTEST.COM (short username kudu)
I20260809 04:43:29.945585 13912 webserver.cc:460] Webserver started at http://127.12.234.1:37291/ using document root <none> and password file <none>
I20260809 04:43:29.946272 13912 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:29.946378 13912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:29.946638 13912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:29.949648 13912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-0/data/instance:
uuid: "13336043a5c549ad8df27ed9be02c046"
format_stamp: "Formatted at 2026-08-09 04:43:29 on dist-test-slave-1jjb"
server_key: "90d855e97472bd8bd38c5e2a1f3089a9"
server_key_iv: "d535304234b57b1ed4f4f96c8d9cb42d"
server_key_version: "encryptionkey@0"
I20260809 04:43:29.950505 13912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance:
uuid: "13336043a5c549ad8df27ed9be02c046"
format_stamp: "Formatted at 2026-08-09 04:43:29 on dist-test-slave-1jjb"
server_key: "90d855e97472bd8bd38c5e2a1f3089a9"
server_key_iv: "d535304234b57b1ed4f4f96c8d9cb42d"
server_key_version: "encryptionkey@0"
I20260809 04:43:29.955897 13912 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 04:43:29.959520 13927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:29.960923 13912 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:43:29.961081 13912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "13336043a5c549ad8df27ed9be02c046"
format_stamp: "Formatted at 2026-08-09 04:43:29 on dist-test-slave-1jjb"
server_key: "90d855e97472bd8bd38c5e2a1f3089a9"
server_key_iv: "d535304234b57b1ed4f4f96c8d9cb42d"
server_key_version: "encryptionkey@0"
I20260809 04:43:29.961285 13912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:29.988277 13912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:29.992816 13912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:29.993129 13912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:29.994100 13912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:43:29.995447 13912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:43:29.995533 13912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:29.995683 13912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:43:29.995740 13912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:30.026335 13912 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:36681
I20260809 04:43:30.026381 14043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:36681 every 8 connection(s)
I20260809 04:43:30.027700 13912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:43:30.034687 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 13912
I20260809 04:43:30.034924 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance
I20260809 04:43:30.035605 13224 external_mini_cluster.cc:1428] Setting key baf27fc35e5897a1f9a67400351aa383
Aug 09 04:43:30 dist-test-slave-1jjb krb5kdc[13801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250609, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:30.048668 14046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:30.030366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:33621 (local address 127.12.234.62:42899)
0809 04:43:30.030814 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:30.030819 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:30.031776 (+ 957us) server_negotiation.cc:407] Connection header received
0809 04:43:30.033581 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:30.033585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:30.033654 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:30.033749 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:30.035330 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:30.036170 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:30.037874 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:30.038300 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:30.038396 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:30.042554 (+ 4158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:30.042563 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:30.042571 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:30.042613 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:30.045445 (+ 2832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:30.046382 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:30.046386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:30.046389 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:30.046506 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:30.046976 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:30.046979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:30.046981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:30.047248 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:30.047359 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:30.048242 (+ 883us) server_negotiation.cc:299] Negotiation successful
0809 04:43:30.048417 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":207,"threads_started":1}
I20260809 04:43:30.050258 14047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:30.030952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:42899 (local address 127.12.234.1:33621)
0809 04:43:30.031608 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:30.031655 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:30.033340 (+ 1685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:30.033936 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:30.033942 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:30.034510 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:30.035170 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:30.035189 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:30.036342 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:30.036346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:30.037704 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:30.037726 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:30.038522 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:30.038531 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:30.038877 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:30.039636 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:30.039671 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:30.042337 (+ 2666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:30.045583 (+ 3246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:30.045595 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:30.045614 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:30.046257 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:30.046615 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:30.046622 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:30.046629 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:30.046869 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:30.047367 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:30.047377 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:30.047819 (+ 442us) client_negotiation.cc:769] Sending connection context
0809 04:43:30.048173 (+ 354us) client_negotiation.cc:240] Negotiation successful
0809 04:43:30.048766 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":251,"threads_started":1}
I20260809 04:43:30.051906 14044 heartbeater.cc:347] Connected to a master server at 127.12.234.62:42899
I20260809 04:43:30.052281 14044 heartbeater.cc:464] Registering TS with master...
I20260809 04:43:30.053282 14044 heartbeater.cc:511] Master 127.12.234.62:42899 requested a full tablet report, sending...
I20260809 04:43:30.055567 13848 ts_manager.cc:194] Registered new tserver with Master: 13336043a5c549ad8df27ed9be02c046 (127.12.234.1:36681)
WARNING: no policy specified for kudu/127.12.234.2@KRBTEST.COM; defaulting to no policy
I20260809 04:43:30.057260 13848 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.1@KRBTEST.COM'} at 127.12.234.1:33621
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
I20260809 04:43:30.081795 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:0
--local_ip_for_outbound_sockets=127.12.234.2
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:42899
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:37663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:30.418373 14050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:30.418680 14050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:30.418838 14050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:30.426275 14050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:30.426399 14050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:30.426580 14050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:43:30.436857 14050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:37663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:42899
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:30.438220 14050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:30.439584 14050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:30.453392 14056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:30.453356 14058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:30.453591 14055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:30.454736 14050 server_base.cc:1034] running on GCE node
I20260809 04:43:30.455400 14050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:30.456470 14050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:30.457674 14050 hybrid_clock.cc:648] HybridClock initialized: now 1786250610457639 us; error 39 us; skew 500 ppm
I20260809 04:43:30.458104 14050 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:30 dist-test-slave-1jjb krb5kdc[13801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250610, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:30.461858 14050 init.cc:377] Logged in from keytab as kudu/127.12.234.2@KRBTEST.COM (short username kudu)
I20260809 04:43:30.463445 14050 webserver.cc:460] Webserver started at http://127.12.234.2:44141/ using document root <none> and password file <none>
I20260809 04:43:30.464217 14050 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:30.464368 14050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:30.464723 14050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:30.467932 14050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-1/data/instance:
uuid: "8358c40981ad412bbfa2a122983b2643"
format_stamp: "Formatted at 2026-08-09 04:43:30 on dist-test-slave-1jjb"
server_key: "ae4c6c92e090608c9e56ff55931d5637"
server_key_iv: "2afba61860732278fb077558a1d95e99"
server_key_version: "encryptionkey@0"
I20260809 04:43:30.468835 14050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance:
uuid: "8358c40981ad412bbfa2a122983b2643"
format_stamp: "Formatted at 2026-08-09 04:43:30 on dist-test-slave-1jjb"
server_key: "ae4c6c92e090608c9e56ff55931d5637"
server_key_iv: "2afba61860732278fb077558a1d95e99"
server_key_version: "encryptionkey@0"
I20260809 04:43:30.474576 14050 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 04:43:30.478240 14066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:30.479530 14050 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:43:30.479709 14050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "8358c40981ad412bbfa2a122983b2643"
format_stamp: "Formatted at 2026-08-09 04:43:30 on dist-test-slave-1jjb"
server_key: "ae4c6c92e090608c9e56ff55931d5637"
server_key_iv: "2afba61860732278fb077558a1d95e99"
server_key_version: "encryptionkey@0"
I20260809 04:43:30.479923 14050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:30.515640 14050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:30.519896 14050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:30.520242 14050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:30.521263 14050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:43:30.522610 14050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:43:30.522715 14050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:30.522861 14050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:43:30.522917 14050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:30.558081 14050 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:45343
I20260809 04:43:30.558176 14184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:45343 every 8 connection(s)
I20260809 04:43:30.559363 14050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:43:30.562594 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 14050
I20260809 04:43:30.562814 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance
I20260809 04:43:30.563408 13224 external_mini_cluster.cc:1428] Setting key 846646b8caba4aa6b47cd57fb9377c1d
Aug 09 04:43:30 dist-test-slave-1jjb krb5kdc[13801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250610, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:30.581485 14187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:30.562004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:51541 (local address 127.12.234.62:42899)
0809 04:43:30.562509 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:30.562514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:30.563435 (+ 921us) server_negotiation.cc:407] Connection header received
0809 04:43:30.565635 (+ 2200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:30.565661 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:30.565741 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:30.565871 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:30.567390 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:30.568584 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:30.570163 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:30.570908 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:30.571048 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:30.574647 (+ 3599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:30.574651 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:30.574655 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:30.574685 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:30.577322 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:30.578080 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:30.578091 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:30.578093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:30.578219 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:30.579293 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:30.579299 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:30.579303 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:30.579642 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:30.579840 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:30.580747 (+ 907us) server_negotiation.cc:299] Negotiation successful
0809 04:43:30.581016 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":291,"threads_started":1}
WARNING: no policy specified for kudu/127.12.234.3@KRBTEST.COM; defaulting to no policy
I20260809 04:43:30.582556 14188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:30.562451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:42899 (local address 127.12.234.2:51541)
0809 04:43:30.563279 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:30.563323 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:30.565321 (+ 1998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:30.566103 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:30.566111 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:30.566637 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:30.567211 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:30.567227 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:30.568729 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:30.568734 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:30.570021 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:30.570032 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:30.571110 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:30.571114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:30.571306 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:30.572024 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:30.572046 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:30.574404 (+ 2358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:30.577470 (+ 3066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:30.577481 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:30.577500 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:30.577951 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:30.578394 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:30.578426 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:30.578431 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:30.578971 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:30.579819 (+ 848us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:30.579838 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:30.580327 (+ 489us) client_negotiation.cc:769] Sending connection context
0809 04:43:30.580671 (+ 344us) client_negotiation.cc:240] Negotiation successful
0809 04:43:30.581281 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":360,"threads_started":1}
I20260809 04:43:30.584178 14185 heartbeater.cc:347] Connected to a master server at 127.12.234.62:42899
I20260809 04:43:30.584496 14185 heartbeater.cc:464] Registering TS with master...
I20260809 04:43:30.585199 14185 heartbeater.cc:511] Master 127.12.234.62:42899 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.3@KRBTEST.COM" created.
I20260809 04:43:30.586833 13848 ts_manager.cc:194] Registered new tserver with Master: 8358c40981ad412bbfa2a122983b2643 (127.12.234.2:45343)
I20260809 04:43:30.587666 13848 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.2@KRBTEST.COM'} at 127.12.234.2:51541
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
I20260809 04:43:30.611095 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:0
--local_ip_for_outbound_sockets=127.12.234.3
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:42899
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:37663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:30.958832 14191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:30.959105 14191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:30.959203 14191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:30.966547 14191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:30.966663 14191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:30.966784 14191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:43:30.976649 14191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:37663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:42899
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:30.977943 14191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:30.979342 14191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:30.993237 14197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:30.993325 14196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:30.993351 14200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:30.994081 14191 server_base.cc:1034] running on GCE node
I20260809 04:43:30.994648 14191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:30.995529 14191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:30.996752 14191 hybrid_clock.cc:648] HybridClock initialized: now 1786250610996705 us; error 53 us; skew 500 ppm
I20260809 04:43:30.997085 14191 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:30 dist-test-slave-1jjb krb5kdc[13801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250610, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:31.000273 14191 init.cc:377] Logged in from keytab as kudu/127.12.234.3@KRBTEST.COM (short username kudu)
I20260809 04:43:31.001799 14191 webserver.cc:460] Webserver started at http://127.12.234.3:45755/ using document root <none> and password file <none>
I20260809 04:43:31.002513 14191 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:31.002653 14191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:31.002971 14191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:31.005960 14191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-2/data/instance:
uuid: "dd2c0798810e4e6a90b3c6f25434000b"
format_stamp: "Formatted at 2026-08-09 04:43:30 on dist-test-slave-1jjb"
server_key: "5055d1e3f1e229a7f9b9ac0e400e45f7"
server_key_iv: "cf2b087c9c2ef8f8bc38964b8e146e16"
server_key_version: "encryptionkey@0"
I20260809 04:43:31.006753 14191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance:
uuid: "dd2c0798810e4e6a90b3c6f25434000b"
format_stamp: "Formatted at 2026-08-09 04:43:30 on dist-test-slave-1jjb"
server_key: "5055d1e3f1e229a7f9b9ac0e400e45f7"
server_key_iv: "cf2b087c9c2ef8f8bc38964b8e146e16"
server_key_version: "encryptionkey@0"
I20260809 04:43:31.012130 14191 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 04:43:31.015578 14207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:31.016927 14191 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 04:43:31.017108 14191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "dd2c0798810e4e6a90b3c6f25434000b"
format_stamp: "Formatted at 2026-08-09 04:43:30 on dist-test-slave-1jjb"
server_key: "5055d1e3f1e229a7f9b9ac0e400e45f7"
server_key_iv: "cf2b087c9c2ef8f8bc38964b8e146e16"
server_key_version: "encryptionkey@0"
I20260809 04:43:31.017318 14191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:31.033809 14191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:31.037921 14191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:31.038208 14191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:31.039180 14191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:43:31.040537 14191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:43:31.040634 14191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:31.040791 14191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:43:31.040844 14191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:31.060774 14044 heartbeater.cc:503] Master 127.12.234.62:42899 was elected leader, sending a full tablet report...
I20260809 04:43:31.077636 14191 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:33233
I20260809 04:43:31.077688 14322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:33233 every 8 connection(s)
I20260809 04:43:31.078974 14191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:43:31.081810 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 14191
I20260809 04:43:31.082021 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance
I20260809 04:43:31.082422 13224 external_mini_cluster.cc:1428] Setting key 7a7ffbc9dbc8038dd39386246a246fdd
Aug 09 04:43:31 dist-test-slave-1jjb krb5kdc[13801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250610, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:31.101075 14325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:31.081816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:60499 (local address 127.12.234.62:42899)
0809 04:43:31.084970 (+ 3154us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:31.084974 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:31.084991 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:43:31.086509 (+ 1518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:31.086513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:31.086577 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:31.086664 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:31.088216 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.088985 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.090512 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.090891 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.090982 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:31.094547 (+ 3565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:31.094552 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:31.094555 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:31.094587 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:31.098244 (+ 3657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:31.098951 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:31.098955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:31.098957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:31.099113 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:31.099483 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:31.099486 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:31.099488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:31.099792 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:31.099913 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:31.100619 (+ 706us) server_negotiation.cc:299] Negotiation successful
0809 04:43:31.100793 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3042,"thread_start_us":2916,"threads_started":1}
I20260809 04:43:31.102085 14327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:31.082515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:42899 (local address 127.12.234.3:60499)
0809 04:43:31.083977 (+ 1462us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:31.084044 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:31.086269 (+ 2225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:31.086861 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:31.086868 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:31.087413 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:31.088019 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:31.088036 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.089119 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.089123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:31.090391 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:31.090404 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.091077 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.091082 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:31.091274 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:31.091902 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:31.091923 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:31.094294 (+ 2371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:31.098389 (+ 4095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:31.098397 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:31.098411 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:31.098847 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:31.099204 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:31.099207 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:31.099209 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:31.099385 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:31.099896 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:31.099901 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:31.100290 (+ 389us) client_negotiation.cc:769] Sending connection context
0809 04:43:31.100518 (+ 228us) client_negotiation.cc:240] Negotiation successful
0809 04:43:31.100914 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1138,"thread_start_us":873,"threads_started":1}
I20260809 04:43:31.103473 14323 heartbeater.cc:347] Connected to a master server at 127.12.234.62:42899
I20260809 04:43:31.103814 14323 heartbeater.cc:464] Registering TS with master...
I20260809 04:43:31.104652 14323 heartbeater.cc:511] Master 127.12.234.62:42899 requested a full tablet report, sending...
I20260809 04:43:31.106046 13848 ts_manager.cc:194] Registered new tserver with Master: dd2c0798810e4e6a90b3c6f25434000b (127.12.234.3:33233)
I20260809 04:43:31.106812 13848 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.3@KRBTEST.COM'} at 127.12.234.3:60499
I20260809 04:43:31.110240 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:43:31.129390 14325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:31.117159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40846 (local address 127.12.234.62:42899)
0809 04:43:31.117380 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:31.117389 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:31.117921 (+ 532us) server_negotiation.cc:407] Connection header received
0809 04:43:31.118040 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:31.118044 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:31.118105 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:31.118189 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:31.119762 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.120640 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.121922 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.122319 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.122410 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:31.124628 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:31.124633 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:31.124636 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:31.124664 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:31.127238 (+ 2574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:31.127784 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:31.127787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:31.127789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:31.127900 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:31.128215 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:31.128219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:31.128220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:31.128499 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:31.128750 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:31.128932 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 04:43:31.129113 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 04:43:31.137462 13848 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40846:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:43:31.146126 13848 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:43:31.190304 14339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:31.162860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40222 (local address 127.12.234.2:45343)
0809 04:43:31.174728 (+ 11868us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:31.174740 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:31.177613 (+ 2873us) server_negotiation.cc:407] Connection header received
0809 04:43:31.178687 (+ 1074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:31.178699 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:31.179004 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:31.179218 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:31.180423 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.182667 (+ 2244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.186827 (+ 4160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.188641 (+ 1814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.188787 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:31.189017 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:31.189697 (+ 680us) server_negotiation.cc:299] Negotiation successful
0809 04:43:31.189875 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11606,"thread_start_us":11449,"threads_started":1}
I20260809 04:43:31.190447 14337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:31.162558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:45343 (local address 127.0.0.1:40222)
0809 04:43:31.177468 (+ 14910us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:31.177495 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:31.178418 (+ 923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:31.179525 (+ 1107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:31.179532 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:31.179569 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:31.180164 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:31.180181 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.182889 (+ 2708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.182897 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:31.186559 (+ 3662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:31.186582 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.189203 (+ 2621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.189208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:31.189379 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:31.189496 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 04:43:31.189641 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 04:43:31.189857 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14732,"mutex_wait_us":41,"spinlock_wait_cycles":768,"thread_start_us":12212,"threads_started":1}
I20260809 04:43:31.190539 14338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:31.162301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:33233 (local address 127.0.0.1:32922)
0809 04:43:31.175229 (+ 12928us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:31.175286 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:31.175559 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:31.177488 (+ 1929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:31.177500 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:31.177544 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:31.178531 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:31.178556 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.181500 (+ 2944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.181510 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:31.185396 (+ 3886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:31.185423 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.189323 (+ 3900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.189329 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:31.189570 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:31.189615 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 04:43:31.189716 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 04:43:31.189961 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12671,"spinlock_wait_cycles":98048,"thread_start_us":15865,"threads_started":1}
I20260809 04:43:31.191440 14340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:31.162866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32922 (local address 127.12.234.3:33233)
0809 04:43:31.176627 (+ 13761us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:31.176640 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:31.176685 (+ 45us) server_negotiation.cc:407] Connection header received
0809 04:43:31.176816 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:31.176828 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:31.177178 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:31.177848 (+ 670us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:31.178909 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.181254 (+ 2345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.185648 (+ 4394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.188676 (+ 3028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.188809 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:31.189104 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:31.190883 (+ 1779us) server_negotiation.cc:299] Negotiation successful
0809 04:43:31.191082 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13517,"thread_start_us":18957,"threads_started":1}
I20260809 04:43:31.194679 14116 tablet_service.cc:1467] Processing CreateTablet for tablet 43eec877c6cd422db5ef52a20426daa9 (DEFAULT_TABLE table=test-table [id=3a415c58915348f2a6b03ed4fdcf8b98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:43:31.194792 14255 tablet_service.cc:1467] Processing CreateTablet for tablet 43eec877c6cd422db5ef52a20426daa9 (DEFAULT_TABLE table=test-table [id=3a415c58915348f2a6b03ed4fdcf8b98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:43:31.196199 14116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43eec877c6cd422db5ef52a20426daa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:31.196318 14255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43eec877c6cd422db5ef52a20426daa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:31.200465 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:31.176235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55512 (local address 127.12.234.1:36681)
0809 04:43:31.187239 (+ 11004us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:31.187248 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:31.188393 (+ 1145us) server_negotiation.cc:407] Connection header received
0809 04:43:31.188518 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:31.188526 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:31.188832 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:31.189035 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:31.190219 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.192215 (+ 1996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.197569 (+ 5354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.199217 (+ 1648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.199406 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:31.199671 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:31.199920 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 04:43:31.200113 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10746,"thread_start_us":10987,"threads_started":1}
I20260809 04:43:31.200552 14341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:31.175770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:36681 (local address 127.0.0.1:55512)
0809 04:43:31.188064 (+ 12294us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:31.188119 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:31.188261 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:31.189326 (+ 1065us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:31.189330 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:31.189356 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:31.190014 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:31.190028 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.192386 (+ 2358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.192391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:31.197345 (+ 4954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:31.197365 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.199480 (+ 2115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.199487 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:31.199721 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:31.199758 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 04:43:31.199860 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 04:43:31.200031 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12153,"mutex_wait_us":1,"thread_start_us":11938,"threads_started":1}
I20260809 04:43:31.204267 13976 tablet_service.cc:1467] Processing CreateTablet for tablet 43eec877c6cd422db5ef52a20426daa9 (DEFAULT_TABLE table=test-table [id=3a415c58915348f2a6b03ed4fdcf8b98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:43:31.205646 13976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43eec877c6cd422db5ef52a20426daa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:31.213469 14344 tablet_bootstrap.cc:492] T 43eec877c6cd422db5ef52a20426daa9 P 8358c40981ad412bbfa2a122983b2643: Bootstrap starting.
I20260809 04:43:31.215437 14345 tablet_bootstrap.cc:492] T 43eec877c6cd422db5ef52a20426daa9 P dd2c0798810e4e6a90b3c6f25434000b: Bootstrap starting.
I20260809 04:43:31.218607 14344 tablet_bootstrap.cc:654] T 43eec877c6cd422db5ef52a20426daa9 P 8358c40981ad412bbfa2a122983b2643: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:31.219357 14345 tablet_bootstrap.cc:654] T 43eec877c6cd422db5ef52a20426daa9 P dd2c0798810e4e6a90b3c6f25434000b: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:31.219620 14346 tablet_bootstrap.cc:492] T 43eec877c6cd422db5ef52a20426daa9 P 13336043a5c549ad8df27ed9be02c046: Bootstrap starting.
I20260809 04:43:31.220152 14344 log.cc:824] T 43eec877c6cd422db5ef52a20426daa9 P 8358c40981ad412bbfa2a122983b2643: Log is configured to *not* fsync() on all Append() calls
I20260809 04:43:31.221110 14345 log.cc:824] T 43eec877c6cd422db5ef52a20426daa9 P dd2c0798810e4e6a90b3c6f25434000b: Log is configured to *not* fsync() on all Append() calls
I20260809 04:43:31.223933 14344 tablet_bootstrap.cc:492] T 43eec877c6cd422db5ef52a20426daa9 P 8358c40981ad412bbfa2a122983b2643: No bootstrap required, opened a new log
I20260809 04:43:31.224177 14346 tablet_bootstrap.cc:654] T 43eec877c6cd422db5ef52a20426daa9 P 13336043a5c549ad8df27ed9be02c046: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:31.224398 14344 ts_tablet_manager.cc:1397] T 43eec877c6cd422db5ef52a20426daa9 P 8358c40981ad412bbfa2a122983b2643: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.005s
I20260809 04:43:31.225178 14345 tablet_bootstrap.cc:492] T 43eec877c6cd422db5ef52a20426daa9 P dd2c0798810e4e6a90b3c6f25434000b: No bootstrap required, opened a new log
I20260809 04:43:31.225525 14345 ts_tablet_manager.cc:1397] T 43eec877c6cd422db5ef52a20426daa9 P dd2c0798810e4e6a90b3c6f25434000b: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260809 04:43:31.225643 14346 log.cc:824] T 43eec877c6cd422db5ef52a20426daa9 P 13336043a5c549ad8df27ed9be02c046: Log is configured to *not* fsync() on all Append() calls
I20260809 04:43:31.228753 14346 tablet_bootstrap.cc:492] T 43eec877c6cd422db5ef52a20426daa9 P 13336043a5c549ad8df27ed9be02c046: No bootstrap required, opened a new log
I20260809 04:43:31.229041 14346 ts_tablet_manager.cc:1397] T 43eec877c6cd422db5ef52a20426daa9 P 13336043a5c549ad8df27ed9be02c046: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260809 04:43:31.233027 14344 raft_consensus.cc:348] T 43eec877c6cd422db5ef52a20426daa9 P 8358c40981ad412bbfa2a122983b2643 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2c0798810e4e6a90b3c6f25434000b" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33233 } } peers { permanent_uuid: "8358c40981ad412bbfa2a122983b2643" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45343 } } peers { permanent_uuid: "13336043a5c549ad8df27ed9be02c046" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36681 } }
I20260809 04:43:31.233403 14344 raft_consensus.cc:374] T 43eec877c6cd422db5ef52a20426daa9 P 8358c40981ad412bbfa2a122983b2643 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:31.233541 14344 raft_consensus.cc:729] T 43eec877c6cd422db5ef52a20426daa9 P 8358c40981ad412bbfa2a122983b2643 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8358c40981ad412bbfa2a122983b2643, State: Initialized, Role: FOLLOWER
I20260809 04:43:31.234160 14344 consensus_queue.cc:260] T 43eec877c6cd422db5ef52a20426daa9 P 8358c40981ad412bbfa2a122983b2643 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2c0798810e4e6a90b3c6f25434000b" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33233 } } peers { permanent_uuid: "8358c40981ad412bbfa2a122983b2643" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45343 } } peers { permanent_uuid: "13336043a5c549ad8df27ed9be02c046" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36681 } }
I20260809 04:43:31.236783 14345 raft_consensus.cc:348] T 43eec877c6cd422db5ef52a20426daa9 P dd2c0798810e4e6a90b3c6f25434000b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2c0798810e4e6a90b3c6f25434000b" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33233 } } peers { permanent_uuid: "8358c40981ad412bbfa2a122983b2643" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45343 } } peers { permanent_uuid: "13336043a5c549ad8df27ed9be02c046" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36681 } }
I20260809 04:43:31.237313 14345 raft_consensus.cc:374] T 43eec877c6cd422db5ef52a20426daa9 P dd2c0798810e4e6a90b3c6f25434000b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:31.237481 14345 raft_consensus.cc:729] T 43eec877c6cd422db5ef52a20426daa9 P dd2c0798810e4e6a90b3c6f25434000b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2c0798810e4e6a90b3c6f25434000b, State: Initialized, Role: FOLLOWER
I20260809 04:43:31.237689 14185 heartbeater.cc:503] Master 127.12.234.62:42899 was elected leader, sending a full tablet report...
I20260809 04:43:31.238226 14344 ts_tablet_manager.cc:1428] T 43eec877c6cd422db5ef52a20426daa9 P 8358c40981ad412bbfa2a122983b2643: Time spent starting tablet: real 0.014s user 0.009s sys 0.003s
I20260809 04:43:31.238211 14345 consensus_queue.cc:260] T 43eec877c6cd422db5ef52a20426daa9 P dd2c0798810e4e6a90b3c6f25434000b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2c0798810e4e6a90b3c6f25434000b" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33233 } } peers { permanent_uuid: "8358c40981ad412bbfa2a122983b2643" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45343 } } peers { permanent_uuid: "13336043a5c549ad8df27ed9be02c046" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36681 } }
I20260809 04:43:31.239473 14323 heartbeater.cc:503] Master 127.12.234.62:42899 was elected leader, sending a full tablet report...
I20260809 04:43:31.240165 14345 ts_tablet_manager.cc:1428] T 43eec877c6cd422db5ef52a20426daa9 P dd2c0798810e4e6a90b3c6f25434000b: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260809 04:43:31.241789 14346 raft_consensus.cc:348] T 43eec877c6cd422db5ef52a20426daa9 P 13336043a5c549ad8df27ed9be02c046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2c0798810e4e6a90b3c6f25434000b" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33233 } } peers { permanent_uuid: "8358c40981ad412bbfa2a122983b2643" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45343 } } peers { permanent_uuid: "13336043a5c549ad8df27ed9be02c046" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36681 } }
I20260809 04:43:31.242264 14346 raft_consensus.cc:374] T 43eec877c6cd422db5ef52a20426daa9 P 13336043a5c549ad8df27ed9be02c046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:31.242380 14346 raft_consensus.cc:729] T 43eec877c6cd422db5ef52a20426daa9 P 13336043a5c549ad8df27ed9be02c046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13336043a5c549ad8df27ed9be02c046, State: Initialized, Role: FOLLOWER
I20260809 04:43:31.243023 14346 consensus_queue.cc:260] T 43eec877c6cd422db5ef52a20426daa9 P 13336043a5c549ad8df27ed9be02c046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2c0798810e4e6a90b3c6f25434000b" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33233 } } peers { permanent_uuid: "8358c40981ad412bbfa2a122983b2643" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45343 } } peers { permanent_uuid: "13336043a5c549ad8df27ed9be02c046" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36681 } }
I20260809 04:43:31.245146 14346 ts_tablet_manager.cc:1428] T 43eec877c6cd422db5ef52a20426daa9 P 13336043a5c549ad8df27ed9be02c046: Time spent starting tablet: real 0.016s user 0.013s sys 0.001s
W20260809 04:43:31.280318 14045 tablet.cc:2369] T 43eec877c6cd422db5ef52a20426daa9 P 13336043a5c549ad8df27ed9be02c046: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:43:31.311514 14186 tablet.cc:2369] T 43eec877c6cd422db5ef52a20426daa9 P 8358c40981ad412bbfa2a122983b2643: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:43:31.330932 14324 tablet.cc:2369] T 43eec877c6cd422db5ef52a20426daa9 P dd2c0798810e4e6a90b3c6f25434000b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:43:31.518090 14351 raft_consensus.cc:482] T 43eec877c6cd422db5ef52a20426daa9 P dd2c0798810e4e6a90b3c6f25434000b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:43:31.518440 14351 raft_consensus.cc:504] T 43eec877c6cd422db5ef52a20426daa9 P dd2c0798810e4e6a90b3c6f25434000b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2c0798810e4e6a90b3c6f25434000b" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33233 } } peers { permanent_uuid: "8358c40981ad412bbfa2a122983b2643" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45343 } } peers { permanent_uuid: "13336043a5c549ad8df27ed9be02c046" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36681 } }
I20260809 04:43:31.520675 14351 leader_election.cc:288] T 43eec877c6cd422db5ef52a20426daa9 P dd2c0798810e4e6a90b3c6f25434000b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8358c40981ad412bbfa2a122983b2643 (127.12.234.2:45343), 13336043a5c549ad8df27ed9be02c046 (127.12.234.1:36681)
I20260809 04:43:31.527182 14339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:31.521100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:52933 (local address 127.12.234.2:45343)
0809 04:43:31.521307 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:31.521316 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:31.521339 (+ 23us) server_negotiation.cc:407] Connection header received
0809 04:43:31.521432 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:31.521436 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:31.521520 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:31.521620 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:31.522241 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.523230 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.525181 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.526363 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.526511 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:31.526565 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:31.526896 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 04:43:31.526991 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 04:43:31.527293 14327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:31.520914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:45343 (local address 127.12.234.3:52933)
0809 04:43:31.521098 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:31.521116 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:31.521262 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:31.521815 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:31.521820 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:31.521846 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:31.522092 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:31.522103 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.523353 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.523358 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:31.525023 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:31.525036 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.526504 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.526509 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:31.526716 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:31.526754 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 04:43:31.526853 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 04:43:31.527009 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 04:43:31.528370 14353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:31.521349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:36681 (local address 127.12.234.3:57151)
0809 04:43:31.521950 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:31.521968 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:31.522092 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:31.522471 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:31.522475 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:31.522497 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:31.522772 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:31.522779 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.524088 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.524092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:31.525910 (+ 1818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:31.525927 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.527588 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.527592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:31.527810 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:31.527852 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 04:43:31.527956 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 04:43:31.528050 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"mutex_wait_us":4,"thread_start_us":245,"threads_started":1}
I20260809 04:43:31.528400 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:31.521476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:57151 (local address 127.12.234.1:36681)
0809 04:43:31.521727 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:31.521733 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:31.522073 (+ 340us) server_negotiation.cc:407] Connection header received
0809 04:43:31.522240 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:31.522246 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:31.522327 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:31.522429 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:31.522950 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.523908 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.526064 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.527204 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.527363 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:31.527455 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:31.527997 (+ 542us) server_negotiation.cc:299] Negotiation successful
0809 04:43:31.528165 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 04:43:31.528604 14136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43eec877c6cd422db5ef52a20426daa9" candidate_uuid: "dd2c0798810e4e6a90b3c6f25434000b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8358c40981ad412bbfa2a122983b2643" is_pre_election: true
I20260809 04:43:31.529458 14136 raft_consensus.cc:2454] T 43eec877c6cd422db5ef52a20426daa9 P 8358c40981ad412bbfa2a122983b2643 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2c0798810e4e6a90b3c6f25434000b in term 0.
I20260809 04:43:31.529520 13997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43eec877c6cd422db5ef52a20426daa9" candidate_uuid: "dd2c0798810e4e6a90b3c6f25434000b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13336043a5c549ad8df27ed9be02c046" is_pre_election: true
I20260809 04:43:31.530107 13997 raft_consensus.cc:2454] T 43eec877c6cd422db5ef52a20426daa9 P 13336043a5c549ad8df27ed9be02c046 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd2c0798810e4e6a90b3c6f25434000b in term 0.
I20260809 04:43:31.530334 14210 leader_election.cc:302] T 43eec877c6cd422db5ef52a20426daa9 P dd2c0798810e4e6a90b3c6f25434000b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8358c40981ad412bbfa2a122983b2643, dd2c0798810e4e6a90b3c6f25434000b; no voters:
I20260809 04:43:31.530843 14351 raft_consensus.cc:2781] T 43eec877c6cd422db5ef52a20426daa9 P dd2c0798810e4e6a90b3c6f25434000b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:43:31.531013 14351 raft_consensus.cc:482] T 43eec877c6cd422db5ef52a20426daa9 P dd2c0798810e4e6a90b3c6f25434000b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:43:31.531136 14351 raft_consensus.cc:3037] T 43eec877c6cd422db5ef52a20426daa9 P dd2c0798810e4e6a90b3c6f25434000b [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:31.533561 14351 raft_consensus.cc:504] T 43eec877c6cd422db5ef52a20426daa9 P dd2c0798810e4e6a90b3c6f25434000b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2c0798810e4e6a90b3c6f25434000b" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33233 } } peers { permanent_uuid: "8358c40981ad412bbfa2a122983b2643" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45343 } } peers { permanent_uuid: "13336043a5c549ad8df27ed9be02c046" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36681 } }
I20260809 04:43:31.534183 14351 leader_election.cc:288] T 43eec877c6cd422db5ef52a20426daa9 P dd2c0798810e4e6a90b3c6f25434000b [CANDIDATE]: Term 1 election: Requested vote from peers 8358c40981ad412bbfa2a122983b2643 (127.12.234.2:45343), 13336043a5c549ad8df27ed9be02c046 (127.12.234.1:36681)
I20260809 04:43:31.534678 14136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43eec877c6cd422db5ef52a20426daa9" candidate_uuid: "dd2c0798810e4e6a90b3c6f25434000b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8358c40981ad412bbfa2a122983b2643"
I20260809 04:43:31.534713 13997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43eec877c6cd422db5ef52a20426daa9" candidate_uuid: "dd2c0798810e4e6a90b3c6f25434000b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13336043a5c549ad8df27ed9be02c046"
I20260809 04:43:31.534883 14136 raft_consensus.cc:3037] T 43eec877c6cd422db5ef52a20426daa9 P 8358c40981ad412bbfa2a122983b2643 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:31.534912 13997 raft_consensus.cc:3037] T 43eec877c6cd422db5ef52a20426daa9 P 13336043a5c549ad8df27ed9be02c046 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:31.537366 14136 raft_consensus.cc:2454] T 43eec877c6cd422db5ef52a20426daa9 P 8358c40981ad412bbfa2a122983b2643 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd2c0798810e4e6a90b3c6f25434000b in term 1.
I20260809 04:43:31.537650 13997 raft_consensus.cc:2454] T 43eec877c6cd422db5ef52a20426daa9 P 13336043a5c549ad8df27ed9be02c046 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd2c0798810e4e6a90b3c6f25434000b in term 1.
I20260809 04:43:31.537899 14210 leader_election.cc:302] T 43eec877c6cd422db5ef52a20426daa9 P dd2c0798810e4e6a90b3c6f25434000b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8358c40981ad412bbfa2a122983b2643, dd2c0798810e4e6a90b3c6f25434000b; no voters:
I20260809 04:43:31.538211 14351 raft_consensus.cc:2781] T 43eec877c6cd422db5ef52a20426daa9 P dd2c0798810e4e6a90b3c6f25434000b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:43:31.538666 14351 raft_consensus.cc:686] T 43eec877c6cd422db5ef52a20426daa9 P dd2c0798810e4e6a90b3c6f25434000b [term 1 LEADER]: Becoming Leader. State: Replica: dd2c0798810e4e6a90b3c6f25434000b, State: Running, Role: LEADER
I20260809 04:43:31.539165 14351 consensus_queue.cc:237] T 43eec877c6cd422db5ef52a20426daa9 P dd2c0798810e4e6a90b3c6f25434000b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2c0798810e4e6a90b3c6f25434000b" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33233 } } peers { permanent_uuid: "8358c40981ad412bbfa2a122983b2643" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45343 } } peers { permanent_uuid: "13336043a5c549ad8df27ed9be02c046" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36681 } }
I20260809 04:43:31.544261 13848 catalog_manager.cc:5495] T 43eec877c6cd422db5ef52a20426daa9 P dd2c0798810e4e6a90b3c6f25434000b reported cstate change: term changed from 0 to 1, leader changed from <none> to dd2c0798810e4e6a90b3c6f25434000b (127.12.234.3). New cstate: current_term: 1 leader_uuid: "dd2c0798810e4e6a90b3c6f25434000b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd2c0798810e4e6a90b3c6f25434000b" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8358c40981ad412bbfa2a122983b2643" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13336043a5c549ad8df27ed9be02c046" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36681 } health_report { overall_health: UNKNOWN } } }
Aug 09 04:43:31 dist-test-slave-1jjb krb5kdc[13801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.1@KRBTEST.COM
I20260809 04:43:31.570719 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:31.553317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46692 (local address 127.12.234.1:36681)
0809 04:43:31.553477 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:31.553483 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:31.553993 (+ 510us) server_negotiation.cc:407] Connection header received
0809 04:43:31.554176 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:31.554180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:31.554260 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:31.554362 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:31.556004 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.556894 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.558666 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.559083 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.559195 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:31.564631 (+ 5436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:31.564647 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:31.564664 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:31.564704 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:31.567705 (+ 3001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:31.568521 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:31.568529 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:31.568532 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:31.568668 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:31.569035 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:31.569040 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:31.569044 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:31.569437 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:31.569815 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:31.570117 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 04:43:31.570331 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 04:43:31 dist-test-slave-1jjb krb5kdc[13801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250611, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 04:43:31 dist-test-slave-1jjb krb5kdc[13801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250611, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.234.1@KRBTEST.COM
I20260809 04:43:31.625988 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:31.608537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46704 (local address 127.12.234.1:36681)
0809 04:43:31.608755 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:31.608766 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:31.609166 (+ 400us) server_negotiation.cc:407] Connection header received
0809 04:43:31.609381 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:31.609386 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:31.609451 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:31.609563 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:31.610983 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.611727 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.613191 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.613657 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.613764 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:31.618500 (+ 4736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:31.618511 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:31.618515 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:31.618561 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:31.622049 (+ 3488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:31.623284 (+ 1235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:31.623291 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:31.623294 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:31.623533 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:31.624392 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:31.624396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:31.624399 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:31.624798 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:31.625250 (+ 452us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:31.625415 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 04:43:31.625678 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
Aug 09 04:43:31 dist-test-slave-1jjb krb5kdc[13801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 04:43:31 dist-test-slave-1jjb krb5kdc[13801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.1@KRBTEST.COM
I20260809 04:43:31.687881 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:31.672282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46714 (local address 127.12.234.1:36681)
0809 04:43:31.672555 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:31.672563 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:31.672751 (+ 188us) server_negotiation.cc:407] Connection header received
0809 04:43:31.672847 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:31.672852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:31.672928 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:31.673023 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:31.674264 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.674959 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.676213 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.676675 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.676771 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:31.680947 (+ 4176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:31.680953 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:31.680956 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:31.680994 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:31.684418 (+ 3424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:31.685604 (+ 1186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:31.685608 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:31.685611 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:31.685748 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:31.686278 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:31.686281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:31.686283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:31.686691 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:31.686907 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:31.687273 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 04:43:31.687480 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
W20260809 04:43:31.690237 13955 tablet_service.cc:2908] Rejecting scan request for tablet 43eec877c6cd422db5ef52a20426daa9: Uninitialized: clean time has not yet been initialized
I20260809 04:43:31.694824 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 13912
I20260809 04:43:31.712316 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:31.695765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46720 (local address 127.12.234.1:36681)
0809 04:43:31.695949 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:31.695954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:31.696422 (+ 468us) server_negotiation.cc:407] Connection header received
0809 04:43:31.696583 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:31.696587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:31.696686 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:31.696781 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:31.698329 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.699166 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.700772 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.701312 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.701450 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:31.705371 (+ 3921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:31.705377 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:31.705380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:31.705433 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:31.708689 (+ 3256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:31.709617 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:31.709624 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:31.709627 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:31.710039 (+ 412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:31.710528 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:31.710532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:31.710535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:31.711089 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:31.711476 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:31.711644 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 04:43:31.711901 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 04:43:31.713251 14039 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:43:31.811084 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 13912
I20260809 04:43:31.833559 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 14050
Aug 09 04:43:31 dist-test-slave-1jjb krb5kdc[13801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.2@KRBTEST.COM
I20260809 04:43:31.851202 14339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:31.834288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42626 (local address 127.12.234.2:45343)
0809 04:43:31.834454 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:31.834458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:31.834476 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:43:31.834574 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:31.834578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:31.834657 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:31.834743 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:31.835961 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.836774 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.838263 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.838660 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.838760 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:31.843147 (+ 4387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:31.843167 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:31.843187 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:31.843229 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:31.847314 (+ 4085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:31.848217 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:31.848226 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:31.848229 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:31.848386 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:31.848908 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:31.848917 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:31.848919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:31.849612 (+ 693us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:31.849992 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:31.850538 (+ 546us) server_negotiation.cc:299] Negotiation successful
0809 04:43:31.850816 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 04:43:31.852339 14180 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:43:31.947264 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 14050
W20260809 04:43:31.971403 14210 connection.cc:535] client connection to 127.12.234.2:45343 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 04:43:31.971781 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 14191
W20260809 04:43:31.971909 14210 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 04:43:31.974174 14353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:31.973596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:45343 (local address 127.12.234.3:56157)
0809 04:43:31.973815 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:31.973956 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.2:45343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260809 04:43:31.974560 14210 consensus_peers.cc:487] T 43eec877c6cd422db5ef52a20426daa9 P dd2c0798810e4e6a90b3c6f25434000b -> Peer 8358c40981ad412bbfa2a122983b2643 (127.12.234.2:45343): Couldn't send request to peer 8358c40981ad412bbfa2a122983b2643. Status: Network error: Client connection negotiation failed: client connection to 127.12.234.2:45343: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
Aug 09 04:43:31 dist-test-slave-1jjb krb5kdc[13801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.3@KRBTEST.COM
I20260809 04:43:31.992326 14371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:31.972767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50278 (local address 127.12.234.3:33233)
0809 04:43:31.973317 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:31.973324 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:31.973350 (+ 26us) server_negotiation.cc:407] Connection header received
0809 04:43:31.973425 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:31.973433 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:31.973524 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:31.973664 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:31.975353 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.976480 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.978272 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:31.978827 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:31.979069 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:31.983774 (+ 4705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:31.983796 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:31.983819 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:31.983872 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:31.987904 (+ 4032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:31.988927 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:31.988939 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:31.988941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:31.989243 (+ 302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:31.990110 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:31.990117 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:31.990120 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:31.990719 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:31.991160 (+ 441us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:31.991481 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 04:43:31.991791 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":267,"threads_started":1}
I20260809 04:43:31.993108 14318 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:43:32.089174 14353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:32.088644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:36681 (local address 127.12.234.3:60711)
0809 04:43:32.088833 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:32.088964 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:36681: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 04:43:32.089553 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 14191
I20260809 04:43:32.114161 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 13814
I20260809 04:43:32.115473 13881 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:43:32.225374 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 13814
2026-08-09T04:43:32Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4572 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:43:32 dist-test-slave-1jjb krb5kdc[14376](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:43:32 dist-test-slave-1jjb krb5kdc[14376](info): set up 2 sockets
Aug 09 04:43:32 dist-test-slave-1jjb krb5kdc[14376](info): commencing operation
krb5kdc: starting...
W20260809 04:43:33.374846 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.001s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:43:33 dist-test-slave-1jjb krb5kdc[14376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:43:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:43:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:43:33.578990 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:40905
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:36729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:40905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:33.883711 14389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:33.884012 14389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:33.884188 14389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:33.891592 14389 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:43:33.891701 14389 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 04:43:33.891789 14389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:33.891898 14389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:33.891963 14389 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:43:33.892064 14389 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:43:33.902196 14389 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:36729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:40905
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:40905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:33.903527 14389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:33.904903 14389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:33.916668 14398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:33.916816 14394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:33.917008 14395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:33.917488 14389 server_base.cc:1034] running on GCE node
I20260809 04:43:33.918273 14389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:33.919621 14389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:33.920874 14389 hybrid_clock.cc:648] HybridClock initialized: now 1786250613920783 us; error 47 us; skew 500 ppm
I20260809 04:43:33.921180 14389 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:33 dist-test-slave-1jjb krb5kdc[14376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250613, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:33.924485 14389 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:43:33.925969 14389 webserver.cc:460] Webserver started at http://127.12.234.62:35013/ using document root <none> and password file <none>
I20260809 04:43:33.926697 14389 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:33.926810 14389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:33.927148 14389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:33.930236 14389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "ae5eda3671af44229eb9257c969e2976"
format_stamp: "Formatted at 2026-08-09 04:43:33 on dist-test-slave-1jjb"
server_key: "5381157908aacec6468355f471f7f4fa"
server_key_iv: "8e6170f754b78d3a8f1294fdd32aec3c"
server_key_version: "encryptionkey@0"
I20260809 04:43:33.931039 14389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "ae5eda3671af44229eb9257c969e2976"
format_stamp: "Formatted at 2026-08-09 04:43:33 on dist-test-slave-1jjb"
server_key: "5381157908aacec6468355f471f7f4fa"
server_key_iv: "8e6170f754b78d3a8f1294fdd32aec3c"
server_key_version: "encryptionkey@0"
I20260809 04:43:33.936384 14389 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 04:43:33.939882 14405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:33.941303 14389 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:43:33.941509 14389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "ae5eda3671af44229eb9257c969e2976"
format_stamp: "Formatted at 2026-08-09 04:43:33 on dist-test-slave-1jjb"
server_key: "5381157908aacec6468355f471f7f4fa"
server_key_iv: "8e6170f754b78d3a8f1294fdd32aec3c"
server_key_version: "encryptionkey@0"
I20260809 04:43:33.941731 14389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:33.956640 14389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:33.960865 14389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:33.961149 14389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:33.984062 14389 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:40905
I20260809 04:43:33.984117 14470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:40905 every 8 connection(s)
I20260809 04:43:33.985548 14389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:43:33.989404 14471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:33.994041 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 14389
I20260809 04:43:33.994280 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:43:33.995101 13224 external_mini_cluster.cc:1428] Setting key 79ab3f532280e4ec6ca97fde5bddded0
I20260809 04:43:34.001402 14471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae5eda3671af44229eb9257c969e2976: Bootstrap starting.
I20260809 04:43:34.005225 14471 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae5eda3671af44229eb9257c969e2976: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:34.006424 14471 log.cc:824] T 00000000000000000000000000000000 P ae5eda3671af44229eb9257c969e2976: Log is configured to *not* fsync() on all Append() calls
I20260809 04:43:34.010048 14471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae5eda3671af44229eb9257c969e2976: No bootstrap required, opened a new log
Aug 09 04:43:34 dist-test-slave-1jjb krb5kdc[14376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:34.017943 14471 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae5eda3671af44229eb9257c969e2976 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae5eda3671af44229eb9257c969e2976" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 40905 } }
I20260809 04:43:34.018339 14471 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae5eda3671af44229eb9257c969e2976 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:34.018460 14471 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae5eda3671af44229eb9257c969e2976 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae5eda3671af44229eb9257c969e2976, State: Initialized, Role: FOLLOWER
I20260809 04:43:34.018935 14475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:33.997214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53858 (local address 127.12.234.62:40905)
0809 04:43:33.998041 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:33.998052 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:33.998130 (+ 78us) server_negotiation.cc:407] Connection header received
0809 04:43:33.999922 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:33.999945 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:34.000407 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:34.000743 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:34.002498 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:34.003841 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:34.005642 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:34.006329 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:34.006468 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:34.011265 (+ 4797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:34.011272 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:34.011288 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:34.011333 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:34.014590 (+ 3257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:34.015305 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:34.015311 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:34.015318 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:34.015490 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:34.015879 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:34.015883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:34.015885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:34.016543 (+ 658us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:34.016886 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:34.017299 (+ 413us) server_negotiation.cc:299] Negotiation successful
0809 04:43:34.017726 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":399,"threads_started":1}
I20260809 04:43:34.019085 14471 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae5eda3671af44229eb9257c969e2976 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae5eda3671af44229eb9257c969e2976" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 40905 } }
I20260809 04:43:34.019368 14471 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae5eda3671af44229eb9257c969e2976 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:43:34.019519 14471 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae5eda3671af44229eb9257c969e2976 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:43:34.019706 14471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae5eda3671af44229eb9257c969e2976 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:34.022858 14471 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae5eda3671af44229eb9257c969e2976 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae5eda3671af44229eb9257c969e2976" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 40905 } }
I20260809 04:43:34.023362 14471 leader_election.cc:302] T 00000000000000000000000000000000 P ae5eda3671af44229eb9257c969e2976 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae5eda3671af44229eb9257c969e2976; no voters:
I20260809 04:43:34.023918 14471 leader_election.cc:288] T 00000000000000000000000000000000 P ae5eda3671af44229eb9257c969e2976 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:43:34.024042 14477 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae5eda3671af44229eb9257c969e2976 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:43:34.024853 14477 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae5eda3671af44229eb9257c969e2976 [term 1 LEADER]: Becoming Leader. State: Replica: ae5eda3671af44229eb9257c969e2976, State: Running, Role: LEADER
I20260809 04:43:34.025391 14477 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae5eda3671af44229eb9257c969e2976 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae5eda3671af44229eb9257c969e2976" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 40905 } }
I20260809 04:43:34.026208 14471 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae5eda3671af44229eb9257c969e2976 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:43:34.028550 14481 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae5eda3671af44229eb9257c969e2976 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae5eda3671af44229eb9257c969e2976. Latest consensus state: current_term: 1 leader_uuid: "ae5eda3671af44229eb9257c969e2976" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae5eda3671af44229eb9257c969e2976" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 40905 } } }
I20260809 04:43:34.028496 14478 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae5eda3671af44229eb9257c969e2976 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae5eda3671af44229eb9257c969e2976" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae5eda3671af44229eb9257c969e2976" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 40905 } } }
I20260809 04:43:34.028947 14481 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae5eda3671af44229eb9257c969e2976 [sys.catalog]: This master's current role is: LEADER
I20260809 04:43:34.028964 14478 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae5eda3671af44229eb9257c969e2976 [sys.catalog]: This master's current role is: LEADER
I20260809 04:43:34.032115 14487 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:43:34.037213 14487 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:43:34.047640 14487 catalog_manager.cc:1347] Generated new cluster ID: 86817e7500b142b2be373028a57eb89c
I20260809 04:43:34.047773 14487 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:43:34.086320 14487 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:43:34.087653 14487 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:43:34.097005 14487 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae5eda3671af44229eb9257c969e2976: Generated new TSK 0
I20260809 04:43:34.098094 14487 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
I20260809 04:43:34.157569 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:0
--local_ip_for_outbound_sockets=127.12.234.1
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:40905
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:36729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:34.490304 14507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:34.490599 14507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:34.490700 14507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:34.497766 14507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:34.497879 14507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:34.497963 14507 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:43:34.498045 14507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:43:34.508366 14507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:36729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.234.62:40905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:34.509693 14507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:34.510970 14507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:34.524726 14515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:34.524747 14513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:34.525051 14512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:34.525679 14507 server_base.cc:1034] running on GCE node
I20260809 04:43:34.526400 14507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:34.527537 14507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:34.528789 14507 hybrid_clock.cc:648] HybridClock initialized: now 1786250614528709 us; error 56 us; skew 500 ppm
I20260809 04:43:34.529114 14507 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:34 dist-test-slave-1jjb krb5kdc[14376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250614, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:34.537146 14507 init.cc:377] Logged in from keytab as kudu/127.12.234.1@KRBTEST.COM (short username kudu)
I20260809 04:43:34.538753 14507 webserver.cc:460] Webserver started at http://127.12.234.1:34403/ using document root <none> and password file <none>
I20260809 04:43:34.539450 14507 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:34.539551 14507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:34.539844 14507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:34.543277 14507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-0/data/instance:
uuid: "1c91046a5e034383963fa5f297cbcccb"
format_stamp: "Formatted at 2026-08-09 04:43:34 on dist-test-slave-1jjb"
server_key: "f99f0c095f3c77006d0b3244984fd69c"
server_key_iv: "34fe001bb732c1f3eb2c2eed65e09896"
server_key_version: "encryptionkey@0"
I20260809 04:43:34.544030 14507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance:
uuid: "1c91046a5e034383963fa5f297cbcccb"
format_stamp: "Formatted at 2026-08-09 04:43:34 on dist-test-slave-1jjb"
server_key: "f99f0c095f3c77006d0b3244984fd69c"
server_key_iv: "34fe001bb732c1f3eb2c2eed65e09896"
server_key_version: "encryptionkey@0"
I20260809 04:43:34.549592 14507 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 04:43:34.553081 14523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:34.554427 14507 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:43:34.554577 14507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "1c91046a5e034383963fa5f297cbcccb"
format_stamp: "Formatted at 2026-08-09 04:43:34 on dist-test-slave-1jjb"
server_key: "f99f0c095f3c77006d0b3244984fd69c"
server_key_iv: "34fe001bb732c1f3eb2c2eed65e09896"
server_key_version: "encryptionkey@0"
I20260809 04:43:34.554777 14507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:34.597481 14507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:34.601567 14507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:34.601809 14507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:34.604748 14507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:43:34.604853 14507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:34.605021 14507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:43:34.605095 14507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:43:34 dist-test-slave-1jjb krb5kdc[14376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250614, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:34.628253 14531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:34.605162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:33583 (local address 127.12.234.62:40905)
0809 04:43:34.605654 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:34.605661 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:34.606862 (+ 1201us) server_negotiation.cc:407] Connection header received
0809 04:43:34.609087 (+ 2225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:34.609095 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:34.609202 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:34.609340 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:34.611490 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:34.612813 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:34.615322 (+ 2509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:34.615747 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:34.615894 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:34.621638 (+ 5744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:34.621644 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:34.621647 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:34.621681 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:34.624385 (+ 2704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:34.625523 (+ 1138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:34.625533 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:34.625539 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:34.625706 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:34.626349 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:34.626354 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:34.626358 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:34.626651 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:34.626808 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:34.627612 (+ 804us) server_negotiation.cc:299] Negotiation successful
0809 04:43:34.627840 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":251,"threads_started":1}
I20260809 04:43:34.629438 14533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:34.605921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40905 (local address 127.12.234.1:33583)
0809 04:43:34.606704 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:34.606742 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:34.608673 (+ 1931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:34.609602 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:34.609613 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:34.610384 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:34.611215 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:34.611234 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:34.613026 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:34.613034 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:34.615164 (+ 2130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:34.615181 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:34.616011 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:34.616019 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:34.616371 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:34.617331 (+ 960us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:34.617373 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:34.621344 (+ 3971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:34.624611 (+ 3267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:34.624623 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:34.624645 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:34.625344 (+ 699us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:34.625860 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:34.625867 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:34.625872 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:34.626216 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:34.626791 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:34.626799 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:34.627165 (+ 366us) client_negotiation.cc:769] Sending connection context
0809 04:43:34.627512 (+ 347us) client_negotiation.cc:240] Negotiation successful
0809 04:43:34.628049 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":331,"threads_started":1}
I20260809 04:43:34.654425 14592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:34.638557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40905 (local address 127.12.234.1:58463)
0809 04:43:34.639191 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:34.639207 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:34.639378 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:34.639747 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:34.639752 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:34.640653 (+ 901us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:34.641180 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:34.641198 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:34.642545 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:34.642554 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:34.644104 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:34.644122 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:34.645267 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:34.645274 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:34.645459 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:34.646319 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:34.646339 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:34.647587 (+ 1248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:34.651238 (+ 3651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:34.651244 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:34.651248 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:34.651657 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:34.652037 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:34.652042 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:34.652045 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:34.652209 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:34.652870 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:34.652875 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:34.653174 (+ 299us) client_negotiation.cc:769] Sending connection context
0809 04:43:34.653361 (+ 187us) client_negotiation.cc:240] Negotiation successful
0809 04:43:34.653771 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":348,"threads_started":1}
I20260809 04:43:34.654922 14531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:34.638456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:58463 (local address 127.12.234.62:40905)
0809 04:43:34.638662 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:34.638668 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:34.639295 (+ 627us) server_negotiation.cc:407] Connection header received
0809 04:43:34.639531 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:34.639537 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:34.639620 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:34.639719 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:34.641334 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:34.642404 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:34.644266 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:34.645064 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:34.645199 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:34.647779 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:34.647786 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:34.647790 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:34.647829 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:34.651111 (+ 3282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:34.651780 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:34.651788 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:34.651792 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:34.651920 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:34.652318 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:34.652323 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:34.652326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:34.652763 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:34.652917 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:34.654480 (+ 1563us) server_negotiation.cc:299] Negotiation successful
0809 04:43:34.654700 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:43:34.662184 14507 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:32795
I20260809 04:43:34.662228 14649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:32795 every 8 connection(s)
I20260809 04:43:34.663375 14507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:43:34.665570 14650 heartbeater.cc:347] Connected to a master server at 127.12.234.62:40905
I20260809 04:43:34.665804 14650 heartbeater.cc:464] Registering TS with master...
I20260809 04:43:34.666450 14650 heartbeater.cc:511] Master 127.12.234.62:40905 requested a full tablet report, sending...
I20260809 04:43:34.668414 14422 ts_manager.cc:194] Registered new tserver with Master: 1c91046a5e034383963fa5f297cbcccb (127.12.234.1:32795)
I20260809 04:43:34.669497 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 14507
I20260809 04:43:34.669739 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance
I20260809 04:43:34.669905 14422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.1@KRBTEST.COM'} at 127.12.234.1:33583
I20260809 04:43:34.670425 13224 external_mini_cluster.cc:1428] Setting key d3b5262375165d2a4721186eb265fcb6
WARNING: no policy specified for kudu/127.12.234.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
I20260809 04:43:34.714766 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:0
--local_ip_for_outbound_sockets=127.12.234.2
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:40905
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:36729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:35.051147 14654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:35.051436 14654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:35.051594 14654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:35.059139 14654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:35.059269 14654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:35.059417 14654 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:43:35.059544 14654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:43:35.069548 14654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:36729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.234.62:40905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:35.070919 14654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:35.072269 14654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:35.085795 14660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:35.085827 14662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:35.086362 14659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:35.086833 14654 server_base.cc:1034] running on GCE node
I20260809 04:43:35.087406 14654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:35.088343 14654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:35.089583 14654 hybrid_clock.cc:648] HybridClock initialized: now 1786250615089515 us; error 45 us; skew 500 ppm
I20260809 04:43:35.089879 14654 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:35 dist-test-slave-1jjb krb5kdc[14376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250615, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:35.093092 14654 init.cc:377] Logged in from keytab as kudu/127.12.234.2@KRBTEST.COM (short username kudu)
I20260809 04:43:35.094601 14654 webserver.cc:460] Webserver started at http://127.12.234.2:36415/ using document root <none> and password file <none>
I20260809 04:43:35.095322 14654 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:35.095458 14654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:35.095786 14654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:35.098899 14654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-1/data/instance:
uuid: "1af1e9ee279b4d3c9da36585d1b98fa8"
format_stamp: "Formatted at 2026-08-09 04:43:35 on dist-test-slave-1jjb"
server_key: "aff5a1b71aef4c6457a289af8453cbd1"
server_key_iv: "9bded625e9abd3a0f4ab77130bf31622"
server_key_version: "encryptionkey@0"
I20260809 04:43:35.099730 14654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance:
uuid: "1af1e9ee279b4d3c9da36585d1b98fa8"
format_stamp: "Formatted at 2026-08-09 04:43:35 on dist-test-slave-1jjb"
server_key: "aff5a1b71aef4c6457a289af8453cbd1"
server_key_iv: "9bded625e9abd3a0f4ab77130bf31622"
server_key_version: "encryptionkey@0"
I20260809 04:43:35.105062 14654 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 04:43:35.108807 14670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:35.110102 14654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 04:43:35.110280 14654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "1af1e9ee279b4d3c9da36585d1b98fa8"
format_stamp: "Formatted at 2026-08-09 04:43:35 on dist-test-slave-1jjb"
server_key: "aff5a1b71aef4c6457a289af8453cbd1"
server_key_iv: "9bded625e9abd3a0f4ab77130bf31622"
server_key_version: "encryptionkey@0"
I20260809 04:43:35.110539 14654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:35.161659 14654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:35.166514 14654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:35.166831 14654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:35.169979 14654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:43:35.170104 14654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:35.170285 14654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:43:35.170382 14654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:43:35 dist-test-slave-1jjb krb5kdc[14376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250615, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:35.199277 14681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:35.171035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40905 (local address 127.12.234.2:43765)
0809 04:43:35.172299 (+ 1264us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:35.172360 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:35.174550 (+ 2190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:35.175569 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:35.175585 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:35.176671 (+ 1086us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:35.177918 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:35.177954 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.179467 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.179476 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:35.182530 (+ 3054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:35.182559 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.183668 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.183677 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:35.183998 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:35.184953 (+ 955us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:35.184999 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:35.188717 (+ 3718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:35.193379 (+ 4662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:35.193398 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:35.193414 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:35.193983 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:35.194545 (+ 562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:35.194553 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:35.194560 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:35.194867 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:35.195647 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:35.195663 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:35.196185 (+ 522us) client_negotiation.cc:769] Sending connection context
0809 04:43:35.196631 (+ 446us) client_negotiation.cc:240] Negotiation successful
0809 04:43:35.197316 (+ 685us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":848,"thread_start_us":554,"threads_started":1}
I20260809 04:43:35.200300 14679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:35.170306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:43765 (local address 127.12.234.62:40905)
0809 04:43:35.172311 (+ 2005us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:35.172318 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:35.172488 (+ 170us) server_negotiation.cc:407] Connection header received
0809 04:43:35.174989 (+ 2501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:35.174996 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:35.175090 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:35.175233 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:35.178184 (+ 2951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.179257 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.182704 (+ 3447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.183368 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.183532 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:35.189109 (+ 5577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:35.189118 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:35.189123 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:35.189178 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:35.193208 (+ 4030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:35.194188 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:35.194197 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:35.194203 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:35.194390 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:35.195018 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:35.195024 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:35.195028 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:35.195489 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:35.195692 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:35.199430 (+ 3738us) server_negotiation.cc:299] Negotiation successful
0809 04:43:35.199761 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1862,"thread_start_us":1765,"threads_started":1}
I20260809 04:43:35.215221 14654 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:38947
I20260809 04:43:35.215289 14800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:38947 every 8 connection(s)
I20260809 04:43:35.216878 14654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:43:35.219166 14801 heartbeater.cc:347] Connected to a master server at 127.12.234.62:40905
I20260809 04:43:35.219429 14801 heartbeater.cc:464] Registering TS with master...
I20260809 04:43:35.220220 14801 heartbeater.cc:511] Master 127.12.234.62:40905 requested a full tablet report, sending...
I20260809 04:43:35.221570 14422 ts_manager.cc:194] Registered new tserver with Master: 1af1e9ee279b4d3c9da36585d1b98fa8 (127.12.234.2:38947)
I20260809 04:43:35.222373 14422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.2@KRBTEST.COM'} at 127.12.234.2:43765
I20260809 04:43:35.224185 14679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:35.207878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:54445 (local address 127.12.234.62:40905)
0809 04:43:35.208150 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:35.208159 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:35.208984 (+ 825us) server_negotiation.cc:407] Connection header received
0809 04:43:35.209231 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:35.209237 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:35.209334 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:35.209478 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:35.211216 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.212590 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.214747 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.215490 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.215648 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:35.218405 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:35.218412 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:35.218417 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:35.218461 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:35.221394 (+ 2933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:35.222095 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:35.222102 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:35.222106 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:35.222240 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:35.222647 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:35.222654 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:35.222657 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:35.222990 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:35.223147 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:35.223574 (+ 427us) server_negotiation.cc:299] Negotiation successful
0809 04:43:35.223796 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 04:43:35.224337 14782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:35.207877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40905 (local address 127.12.234.2:54445)
0809 04:43:35.208842 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:35.208864 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:35.209063 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:35.209526 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:35.209535 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:35.210316 (+ 781us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:35.211012 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:35.211030 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.212784 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.212793 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:35.214572 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:35.214595 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.215773 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.215780 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:35.215937 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:35.216731 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:35.216749 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:35.217890 (+ 1141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:35.221527 (+ 3637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:35.221533 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:35.221538 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:35.221978 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:35.222351 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:35.222363 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:35.222367 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:35.222534 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:35.223113 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:35.223118 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:35.223305 (+ 187us) client_negotiation.cc:769] Sending connection context
0809 04:43:35.223457 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 04:43:35.223817 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"thread_start_us":490,"threads_started":1}
I20260809 04:43:35.226089 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 14654
I20260809 04:43:35.226289 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance
I20260809 04:43:35.226748 13224 external_mini_cluster.cc:1428] Setting key 85df8b9d30c5664e7d88a385ae79e1fb
WARNING: no policy specified for kudu/127.12.234.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
I20260809 04:43:35.271317 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:0
--local_ip_for_outbound_sockets=127.12.234.3
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:40905
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:36729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:35.619591 14805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:35.619903 14805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:35.620057 14805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:35.627363 14805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:35.627499 14805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:35.627629 14805 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:43:35.627755 14805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:43:35.637888 14805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:36729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.234.62:40905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:35.639197 14805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:35.640574 14805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:35.653947 14810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:35.654042 14811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:35.654078 14813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:35.654950 14805 server_base.cc:1034] running on GCE node
I20260809 04:43:35.655529 14805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:35.656481 14805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:35.657737 14805 hybrid_clock.cc:648] HybridClock initialized: now 1786250615657656 us; error 51 us; skew 500 ppm
I20260809 04:43:35.658025 14805 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:35 dist-test-slave-1jjb krb5kdc[14376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250615, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:35.661338 14805 init.cc:377] Logged in from keytab as kudu/127.12.234.3@KRBTEST.COM (short username kudu)
I20260809 04:43:35.663017 14805 webserver.cc:460] Webserver started at http://127.12.234.3:39121/ using document root <none> and password file <none>
I20260809 04:43:35.663726 14805 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:35.663859 14805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:35.664242 14805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:35.667098 14805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-2/data/instance:
uuid: "d3397185e1f3439b9658a61e0992de39"
format_stamp: "Formatted at 2026-08-09 04:43:35 on dist-test-slave-1jjb"
server_key: "aca7673686f029cfd57bdbedb848c797"
server_key_iv: "2ca676aec7708455f4f4f4dee7b0ef7e"
server_key_version: "encryptionkey@0"
I20260809 04:43:35.667889 14805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance:
uuid: "d3397185e1f3439b9658a61e0992de39"
format_stamp: "Formatted at 2026-08-09 04:43:35 on dist-test-slave-1jjb"
server_key: "aca7673686f029cfd57bdbedb848c797"
server_key_iv: "2ca676aec7708455f4f4f4dee7b0ef7e"
server_key_version: "encryptionkey@0"
I20260809 04:43:35.672824 14650 heartbeater.cc:503] Master 127.12.234.62:40905 was elected leader, sending a full tablet report...
I20260809 04:43:35.673122 14805 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 04:43:35.676802 14820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:35.678109 14805 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:43:35.678287 14805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "d3397185e1f3439b9658a61e0992de39"
format_stamp: "Formatted at 2026-08-09 04:43:35 on dist-test-slave-1jjb"
server_key: "aca7673686f029cfd57bdbedb848c797"
server_key_iv: "2ca676aec7708455f4f4f4dee7b0ef7e"
server_key_version: "encryptionkey@0"
I20260809 04:43:35.678517 14805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:35.724160 14805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:35.729024 14805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:35.729355 14805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:35.732301 14805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:43:35.732424 14805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:35.732568 14805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:43:35.732638 14805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:43:35 dist-test-slave-1jjb krb5kdc[14376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250615, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:35.766460 14830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:35.733818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40905 (local address 127.12.234.3:51675)
0809 04:43:35.735018 (+ 1200us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:35.735073 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:35.738428 (+ 3355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:35.745366 (+ 6938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:35.745384 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:35.746389 (+ 1005us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:35.747515 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:35.747548 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.748871 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.748880 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:35.752061 (+ 3181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:35.752255 (+ 194us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.753135 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.753145 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:35.753530 (+ 385us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:35.754475 (+ 945us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:35.754517 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:35.758745 (+ 4228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:35.762303 (+ 3558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:35.762313 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:35.762333 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:35.762799 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:35.763224 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:35.763229 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:35.763233 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:35.763441 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:35.763981 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:35.763989 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:35.764483 (+ 494us) client_negotiation.cc:769] Sending connection context
0809 04:43:35.764882 (+ 399us) client_negotiation.cc:240] Negotiation successful
0809 04:43:35.765339 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":821,"thread_start_us":541,"threads_started":1}
I20260809 04:43:35.767567 14829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:35.732955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:51675 (local address 127.12.234.62:40905)
0809 04:43:35.744520 (+ 11565us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:35.744528 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:35.744558 (+ 30us) server_negotiation.cc:407] Connection header received
0809 04:43:35.744643 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:35.744651 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:35.744755 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:35.744897 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:35.747767 (+ 2870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.748684 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.752418 (+ 3734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.752850 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.752964 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:35.759108 (+ 6144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:35.759114 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:35.759117 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:35.759155 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:35.762148 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:35.762947 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:35.762952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:35.762954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:35.763100 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:35.763565 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:35.763569 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:35.763571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:35.763837 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:35.763991 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:35.767112 (+ 3121us) server_negotiation.cc:299] Negotiation successful
0809 04:43:35.767285 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11410,"thread_start_us":11301,"threads_started":1}
I20260809 04:43:35.792793 14829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:35.775623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:49421 (local address 127.12.234.62:40905)
0809 04:43:35.775885 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:35.775895 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:35.776590 (+ 695us) server_negotiation.cc:407] Connection header received
0809 04:43:35.776877 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:35.776887 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:35.777001 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:35.777144 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:35.778125 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.779590 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.781365 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.782211 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.782352 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:35.785407 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:35.785412 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:35.785416 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:35.785448 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:35.789166 (+ 3718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:35.790241 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:35.790248 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:35.790254 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:35.790397 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:35.790958 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:35.790967 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:35.790974 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:35.791412 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:35.791588 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:35.792200 (+ 612us) server_negotiation.cc:299] Negotiation successful
0809 04:43:35.792445 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 04:43:35.792780 14906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:35.775639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40905 (local address 127.12.234.3:49421)
0809 04:43:35.776455 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:35.776477 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:35.776692 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:35.777202 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:35.777206 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:35.777671 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:35.777973 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:35.777982 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.779728 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.779731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:35.781163 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:35.781180 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.782693 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.782703 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:35.782978 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:35.783879 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:35.783905 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:35.785194 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:35.789326 (+ 4132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:35.789343 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:35.789348 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:35.790082 (+ 734us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:35.790580 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:35.790583 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:35.790589 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:35.790809 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:35.791577 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:35.791584 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:35.791847 (+ 263us) client_negotiation.cc:769] Sending connection context
0809 04:43:35.792031 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 04:43:35.792424 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":398,"threads_started":1}
I20260809 04:43:35.794844 14805 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:37625
I20260809 04:43:35.794924 14947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:37625 every 8 connection(s)
I20260809 04:43:35.796164 14805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:43:35.798448 14948 heartbeater.cc:347] Connected to a master server at 127.12.234.62:40905
I20260809 04:43:35.798697 14948 heartbeater.cc:464] Registering TS with master...
I20260809 04:43:35.799356 14948 heartbeater.cc:511] Master 127.12.234.62:40905 requested a full tablet report, sending...
I20260809 04:43:35.800962 14422 ts_manager.cc:194] Registered new tserver with Master: d3397185e1f3439b9658a61e0992de39 (127.12.234.3:37625)
I20260809 04:43:35.801844 14422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.3@KRBTEST.COM'} at 127.12.234.3:51675
I20260809 04:43:35.803439 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 14805
I20260809 04:43:35.803638 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance
I20260809 04:43:35.804207 13224 external_mini_cluster.cc:1428] Setting key 868d4d1cacda03e5ff51f1c79262edbd
I20260809 04:43:35.807030 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 04:43:35 dist-test-slave-1jjb krb5kdc[14376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250615, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 04:43:35 dist-test-slave-1jjb krb5kdc[14376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250615, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:35.858878 14829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:35.844760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53862 (local address 127.12.234.62:40905)
0809 04:43:35.845036 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:35.845043 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:35.845247 (+ 204us) server_negotiation.cc:407] Connection header received
0809 04:43:35.845402 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:35.845408 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:35.845550 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:35.845648 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:35.847084 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.847842 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.849199 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.849646 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.849764 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:35.853592 (+ 3828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:35.853597 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:35.853601 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:35.853638 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:35.856299 (+ 2661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:35.856869 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:35.856873 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:35.856875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:35.857010 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:35.857342 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:35.857346 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:35.857348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:35.857622 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:35.857859 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:35.858255 (+ 396us) server_negotiation.cc:299] Negotiation successful
0809 04:43:35.858463 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 04:43:35.863329 14422 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53862:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:43:35.870379 14422 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:43:35.898664 14964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:35.884012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45352 (local address 127.12.234.3:37625)
0809 04:43:35.884841 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:35.884854 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:35.889474 (+ 4620us) server_negotiation.cc:407] Connection header received
0809 04:43:35.889803 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:35.889812 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:35.890077 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:35.890262 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:35.891133 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.892904 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.895828 (+ 2924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.897385 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.897539 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:35.897771 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:35.898103 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 04:43:35.898262 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":432,"threads_started":1}
I20260809 04:43:35.898674 14961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:35.883390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:37625 (local address 127.0.0.1:45352)
0809 04:43:35.889344 (+ 5954us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:35.889362 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:35.889593 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:35.890598 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:35.890604 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:35.890627 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:35.890926 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:35.890935 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.893083 (+ 2148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.893095 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:35.895677 (+ 2582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:35.895689 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.897619 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.897625 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:35.897830 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:35.897881 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 04:43:35.898014 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 04:43:35.898211 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5829,"mutex_wait_us":1,"thread_start_us":4893,"threads_started":1}
I20260809 04:43:35.900415 14962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:35.883011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42838 (local address 127.12.234.2:38947)
0809 04:43:35.891527 (+ 8516us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:35.891535 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:35.891563 (+ 28us) server_negotiation.cc:407] Connection header received
0809 04:43:35.891658 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:35.891666 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:35.891878 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:35.892052 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:35.893698 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.895229 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.897268 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.899098 (+ 1830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.899230 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:35.899445 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:35.899900 (+ 455us) server_negotiation.cc:299] Negotiation successful
0809 04:43:35.900094 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8310,"thread_start_us":9579,"threads_started":1}
I20260809 04:43:35.900470 14963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:35.882860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:38947 (local address 127.0.0.1:42838)
0809 04:43:35.888692 (+ 5832us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:35.888729 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:35.888896 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:35.892919 (+ 4023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:35.892923 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:35.892948 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:35.893501 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:35.893515 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.895374 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.895380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:35.897114 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:35.897126 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.899584 (+ 2458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.899591 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:35.899721 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:35.899744 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 04:43:35.899840 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 04:43:35.899977 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5584,"mutex_wait_us":30,"thread_start_us":6284,"threads_started":1}
I20260809 04:43:35.901641 14965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:35.889697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:32795 (local address 127.0.0.1:60202)
0809 04:43:35.892671 (+ 2974us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:35.892688 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:35.892788 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:35.893910 (+ 1122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:35.893914 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:35.893947 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:35.894483 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:35.894497 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.896280 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.896288 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:35.898488 (+ 2200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:35.898502 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.900747 (+ 2245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.900753 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:35.900948 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:35.900979 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 04:43:35.901058 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 04:43:35.901193 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2844,"mutex_wait_us":10,"thread_start_us":2709,"threads_started":1}
I20260809 04:43:35.902168 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:35.890044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60202 (local address 127.12.234.1:32795)
0809 04:43:35.893203 (+ 3159us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:35.893209 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:35.893230 (+ 21us) server_negotiation.cc:407] Connection header received
0809 04:43:35.893303 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:35.893307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:35.893505 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:35.893699 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:35.894691 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.896133 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.898695 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:35.900327 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:35.900472 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:35.901554 (+ 1082us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:35.901622 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 04:43:35.901792 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3015,"thread_start_us":2876,"threads_started":1}
I20260809 04:43:35.903177 14872 tablet_service.cc:1467] Processing CreateTablet for tablet 471e79cbe15a480da83ece93f5934c7b (DEFAULT_TABLE table=test-table [id=6d0e31559b7a433fbf65f1df26fa7a1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:43:35.903688 14726 tablet_service.cc:1467] Processing CreateTablet for tablet 471e79cbe15a480da83ece93f5934c7b (DEFAULT_TABLE table=test-table [id=6d0e31559b7a433fbf65f1df26fa7a1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:43:35.904873 14872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 471e79cbe15a480da83ece93f5934c7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:35.905503 14726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 471e79cbe15a480da83ece93f5934c7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:35.905486 14578 tablet_service.cc:1467] Processing CreateTablet for tablet 471e79cbe15a480da83ece93f5934c7b (DEFAULT_TABLE table=test-table [id=6d0e31559b7a433fbf65f1df26fa7a1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:43:35.906772 14578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 471e79cbe15a480da83ece93f5934c7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:35.917233 14967 tablet_bootstrap.cc:492] T 471e79cbe15a480da83ece93f5934c7b P 1af1e9ee279b4d3c9da36585d1b98fa8: Bootstrap starting.
I20260809 04:43:35.921427 14967 tablet_bootstrap.cc:654] T 471e79cbe15a480da83ece93f5934c7b P 1af1e9ee279b4d3c9da36585d1b98fa8: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:35.920791 14968 tablet_bootstrap.cc:492] T 471e79cbe15a480da83ece93f5934c7b P d3397185e1f3439b9658a61e0992de39: Bootstrap starting.
I20260809 04:43:35.922995 14969 tablet_bootstrap.cc:492] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb: Bootstrap starting.
I20260809 04:43:35.923218 14967 log.cc:824] T 471e79cbe15a480da83ece93f5934c7b P 1af1e9ee279b4d3c9da36585d1b98fa8: Log is configured to *not* fsync() on all Append() calls
I20260809 04:43:35.925475 14968 tablet_bootstrap.cc:654] T 471e79cbe15a480da83ece93f5934c7b P d3397185e1f3439b9658a61e0992de39: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:35.926352 14969 tablet_bootstrap.cc:654] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:35.926903 14968 log.cc:824] T 471e79cbe15a480da83ece93f5934c7b P d3397185e1f3439b9658a61e0992de39: Log is configured to *not* fsync() on all Append() calls
I20260809 04:43:35.927034 14967 tablet_bootstrap.cc:492] T 471e79cbe15a480da83ece93f5934c7b P 1af1e9ee279b4d3c9da36585d1b98fa8: No bootstrap required, opened a new log
I20260809 04:43:35.927459 14969 log.cc:824] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb: Log is configured to *not* fsync() on all Append() calls
I20260809 04:43:35.927484 14967 ts_tablet_manager.cc:1397] T 471e79cbe15a480da83ece93f5934c7b P 1af1e9ee279b4d3c9da36585d1b98fa8: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260809 04:43:35.930346 14968 tablet_bootstrap.cc:492] T 471e79cbe15a480da83ece93f5934c7b P d3397185e1f3439b9658a61e0992de39: No bootstrap required, opened a new log
I20260809 04:43:35.930480 14969 tablet_bootstrap.cc:492] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb: No bootstrap required, opened a new log
I20260809 04:43:35.930640 14968 ts_tablet_manager.cc:1397] T 471e79cbe15a480da83ece93f5934c7b P d3397185e1f3439b9658a61e0992de39: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260809 04:43:35.930769 14969 ts_tablet_manager.cc:1397] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260809 04:43:35.936136 14967 raft_consensus.cc:348] T 471e79cbe15a480da83ece93f5934c7b P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } }
I20260809 04:43:35.936529 14967 raft_consensus.cc:374] T 471e79cbe15a480da83ece93f5934c7b P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:35.936631 14967 raft_consensus.cc:729] T 471e79cbe15a480da83ece93f5934c7b P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1af1e9ee279b4d3c9da36585d1b98fa8, State: Initialized, Role: FOLLOWER
I20260809 04:43:35.937273 14967 consensus_queue.cc:260] T 471e79cbe15a480da83ece93f5934c7b P 1af1e9ee279b4d3c9da36585d1b98fa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } }
I20260809 04:43:35.938374 14801 heartbeater.cc:503] Master 127.12.234.62:40905 was elected leader, sending a full tablet report...
I20260809 04:43:35.939079 14967 ts_tablet_manager.cc:1428] T 471e79cbe15a480da83ece93f5934c7b P 1af1e9ee279b4d3c9da36585d1b98fa8: Time spent starting tablet: real 0.011s user 0.009s sys 0.004s
I20260809 04:43:35.941970 14969 raft_consensus.cc:348] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } }
I20260809 04:43:35.941970 14968 raft_consensus.cc:348] T 471e79cbe15a480da83ece93f5934c7b P d3397185e1f3439b9658a61e0992de39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } }
I20260809 04:43:35.942426 14968 raft_consensus.cc:374] T 471e79cbe15a480da83ece93f5934c7b P d3397185e1f3439b9658a61e0992de39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:35.942423 14969 raft_consensus.cc:374] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:35.942543 14969 raft_consensus.cc:729] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c91046a5e034383963fa5f297cbcccb, State: Initialized, Role: FOLLOWER
I20260809 04:43:35.942533 14968 raft_consensus.cc:729] T 471e79cbe15a480da83ece93f5934c7b P d3397185e1f3439b9658a61e0992de39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3397185e1f3439b9658a61e0992de39, State: Initialized, Role: FOLLOWER
I20260809 04:43:35.943182 14968 consensus_queue.cc:260] T 471e79cbe15a480da83ece93f5934c7b P d3397185e1f3439b9658a61e0992de39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } }
I20260809 04:43:35.943182 14969 consensus_queue.cc:260] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } }
I20260809 04:43:35.945268 14969 ts_tablet_manager.cc:1428] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb: Time spent starting tablet: real 0.014s user 0.012s sys 0.002s
I20260809 04:43:35.945359 14948 heartbeater.cc:503] Master 127.12.234.62:40905 was elected leader, sending a full tablet report...
I20260809 04:43:35.948253 14968 ts_tablet_manager.cc:1428] T 471e79cbe15a480da83ece93f5934c7b P d3397185e1f3439b9658a61e0992de39: Time spent starting tablet: real 0.018s user 0.017s sys 0.000s
W20260809 04:43:35.968976 14802 tablet.cc:2369] T 471e79cbe15a480da83ece93f5934c7b P 1af1e9ee279b4d3c9da36585d1b98fa8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:43:36.048561 14949 tablet.cc:2369] T 471e79cbe15a480da83ece93f5934c7b P d3397185e1f3439b9658a61e0992de39: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:43:36.093947 14974 raft_consensus.cc:482] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:43:36.094319 14974 raft_consensus.cc:504] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } }
I20260809 04:43:36.096067 14974 leader_election.cc:288] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1af1e9ee279b4d3c9da36585d1b98fa8 (127.12.234.2:38947), d3397185e1f3439b9658a61e0992de39 (127.12.234.3:37625)
I20260809 04:43:36.105284 14964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:36.096588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:53913 (local address 127.12.234.3:37625)
0809 04:43:36.096798 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:36.096805 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:36.097609 (+ 804us) server_negotiation.cc:407] Connection header received
0809 04:43:36.097763 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:36.097768 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:36.097839 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:36.097942 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:36.098777 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.099925 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.102912 (+ 2987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.104215 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.104395 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:36.104487 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:36.104875 (+ 388us) server_negotiation.cc:299] Negotiation successful
0809 04:43:36.105010 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 04:43:36.105293 14962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:36.096729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:41109 (local address 127.12.234.2:38947)
0809 04:43:36.096930 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:36.096937 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:36.097494 (+ 557us) server_negotiation.cc:407] Connection header received
0809 04:43:36.097753 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:36.097758 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:36.097839 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:36.097943 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:36.098732 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.099976 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.103003 (+ 3027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.104351 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.104496 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:36.104567 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:36.104888 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 04:43:36.105018 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:43:36.105522 14977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:36.096386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:38947 (local address 127.12.234.1:41109)
0809 04:43:36.097375 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:36.097395 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:36.097579 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:36.097973 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:36.097978 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:36.098005 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:36.098527 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:36.098546 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.100113 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.100118 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:36.102860 (+ 2742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:36.102874 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.104569 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.104575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:36.104725 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:36.104749 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 04:43:36.104840 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 04:43:36.105044 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":837,"mutex_wait_us":140,"thread_start_us":757,"threads_started":1}
I20260809 04:43:36.105896 14976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:36.096438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:37625 (local address 127.12.234.1:53913)
0809 04:43:36.097518 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:36.097534 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:36.097637 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:36.097973 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:36.097978 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:36.098064 (+ 86us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:36.098594 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:36.098608 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.100119 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.100123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:36.102753 (+ 2630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:36.102770 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.104467 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.104477 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:36.104692 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:36.104731 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 04:43:36.104841 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 04:43:36.105044 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":934,"mutex_wait_us":155,"spinlock_wait_cycles":22912,"thread_start_us":568,"threads_started":1}
I20260809 04:43:36.106761 14747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "471e79cbe15a480da83ece93f5934c7b" candidate_uuid: "1c91046a5e034383963fa5f297cbcccb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" is_pre_election: true
I20260809 04:43:36.106920 14899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "471e79cbe15a480da83ece93f5934c7b" candidate_uuid: "1c91046a5e034383963fa5f297cbcccb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3397185e1f3439b9658a61e0992de39" is_pre_election: true
I20260809 04:43:36.107362 14747 raft_consensus.cc:2454] T 471e79cbe15a480da83ece93f5934c7b P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c91046a5e034383963fa5f297cbcccb in term 0.
I20260809 04:43:36.107456 14899 raft_consensus.cc:2454] T 471e79cbe15a480da83ece93f5934c7b P d3397185e1f3439b9658a61e0992de39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c91046a5e034383963fa5f297cbcccb in term 0.
I20260809 04:43:36.108191 14526 leader_election.cc:302] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1af1e9ee279b4d3c9da36585d1b98fa8, 1c91046a5e034383963fa5f297cbcccb, d3397185e1f3439b9658a61e0992de39; no voters:
I20260809 04:43:36.108664 14974 raft_consensus.cc:2781] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:43:36.108809 14974 raft_consensus.cc:482] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:43:36.108922 14974 raft_consensus.cc:3037] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:36.111275 14974 raft_consensus.cc:504] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } }
I20260809 04:43:36.111902 14974 leader_election.cc:288] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb [CANDIDATE]: Term 1 election: Requested vote from peers 1af1e9ee279b4d3c9da36585d1b98fa8 (127.12.234.2:38947), d3397185e1f3439b9658a61e0992de39 (127.12.234.3:37625)
I20260809 04:43:36.112387 14899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "471e79cbe15a480da83ece93f5934c7b" candidate_uuid: "1c91046a5e034383963fa5f297cbcccb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3397185e1f3439b9658a61e0992de39"
I20260809 04:43:36.112391 14747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "471e79cbe15a480da83ece93f5934c7b" candidate_uuid: "1c91046a5e034383963fa5f297cbcccb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8"
I20260809 04:43:36.112612 14899 raft_consensus.cc:3037] T 471e79cbe15a480da83ece93f5934c7b P d3397185e1f3439b9658a61e0992de39 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:36.112623 14747 raft_consensus.cc:3037] T 471e79cbe15a480da83ece93f5934c7b P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:36.114956 14899 raft_consensus.cc:2454] T 471e79cbe15a480da83ece93f5934c7b P d3397185e1f3439b9658a61e0992de39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c91046a5e034383963fa5f297cbcccb in term 1.
I20260809 04:43:36.115115 14747 raft_consensus.cc:2454] T 471e79cbe15a480da83ece93f5934c7b P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c91046a5e034383963fa5f297cbcccb in term 1.
I20260809 04:43:36.115502 14524 leader_election.cc:302] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c91046a5e034383963fa5f297cbcccb, d3397185e1f3439b9658a61e0992de39; no voters:
I20260809 04:43:36.115823 14974 raft_consensus.cc:2781] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:43:36.116353 14974 raft_consensus.cc:686] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb [term 1 LEADER]: Becoming Leader. State: Replica: 1c91046a5e034383963fa5f297cbcccb, State: Running, Role: LEADER
I20260809 04:43:36.116830 14974 consensus_queue.cc:237] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } }
I20260809 04:43:36.121722 14421 catalog_manager.cc:5495] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c91046a5e034383963fa5f297cbcccb (127.12.234.1). New cstate: current_term: 1 leader_uuid: "1c91046a5e034383963fa5f297cbcccb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } health_report { overall_health: UNKNOWN } } }
W20260809 04:43:36.165959 14651 tablet.cc:2369] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:43:36.202744 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:36.197369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60214 (local address 127.12.234.1:32795)
0809 04:43:36.197565 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:36.197569 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:36.197587 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:43:36.197675 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:36.197678 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:36.197752 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:36.197865 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:43:36.198213 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.198982 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.200543 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.200929 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.201021 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:36.201463 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:43:36.202123 (+ 660us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:43:36.202237 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:36.202335 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 04:43:36.202403 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:43:36.212000 14899 raft_consensus.cc:1261] T 471e79cbe15a480da83ece93f5934c7b P d3397185e1f3439b9658a61e0992de39 [term 1 FOLLOWER]: Refusing update from remote peer 1c91046a5e034383963fa5f297cbcccb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:43:36.213129 14978 consensus_queue.cc:1035] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:43:36.213443 14747 raft_consensus.cc:1261] T 471e79cbe15a480da83ece93f5934c7b P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 1 FOLLOWER]: Refusing update from remote peer 1c91046a5e034383963fa5f297cbcccb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:43:36.214260 14978 consensus_queue.cc:1035] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:43:36.251724 14421 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:43:36.252199 14421 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:43:36.256527 14421 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ae5eda3671af44229eb9257c969e2976: Sending DeleteTablet for 3 replicas of tablet 471e79cbe15a480da83ece93f5934c7b
I20260809 04:43:36.257953 14578 tablet_service.cc:1514] Processing DeleteTablet for tablet 471e79cbe15a480da83ece93f5934c7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:43:36 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:60202
I20260809 04:43:36.258016 14872 tablet_service.cc:1514] Processing DeleteTablet for tablet 471e79cbe15a480da83ece93f5934c7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:43:36 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:45352
I20260809 04:43:36.258059 14726 tablet_service.cc:1514] Processing DeleteTablet for tablet 471e79cbe15a480da83ece93f5934c7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:43:36 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:42838
I20260809 04:43:36.264063 14421 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53862:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:43:36.264940 14421 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:43:36.270357 14992 tablet_replica.cc:330] T 471e79cbe15a480da83ece93f5934c7b P 1af1e9ee279b4d3c9da36585d1b98fa8: stopping tablet replica
I20260809 04:43:36.271034 14992 raft_consensus.cc:2229] T 471e79cbe15a480da83ece93f5934c7b P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:43:36.271613 14992 raft_consensus.cc:2258] T 471e79cbe15a480da83ece93f5934c7b P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:43:36.272599 14991 tablet_replica.cc:330] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb: stopping tablet replica
I20260809 04:43:36.273063 14991 raft_consensus.cc:2229] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:43:36.273981 14991 raft_consensus.cc:2258] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:43:36.275017 14577 tablet_service.cc:1467] Processing CreateTablet for tablet b44efea240144200a816936377674b72 (DEFAULT_TABLE table=test-table [id=b63ccc48b1504e72a3f19311b5760322]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:43:36.275179 14726 tablet_service.cc:1467] Processing CreateTablet for tablet b44efea240144200a816936377674b72 (DEFAULT_TABLE table=test-table [id=b63ccc48b1504e72a3f19311b5760322]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:43:36.275367 14871 tablet_service.cc:1467] Processing CreateTablet for tablet b44efea240144200a816936377674b72 (DEFAULT_TABLE table=test-table [id=b63ccc48b1504e72a3f19311b5760322]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:43:36.275513 14577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b44efea240144200a816936377674b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:36.275657 14726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b44efea240144200a816936377674b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:36.275830 14871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b44efea240144200a816936377674b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:36.280143 14969 tablet_bootstrap.cc:492] T b44efea240144200a816936377674b72 P 1c91046a5e034383963fa5f297cbcccb: Bootstrap starting.
I20260809 04:43:36.282253 14969 tablet_bootstrap.cc:654] T b44efea240144200a816936377674b72 P 1c91046a5e034383963fa5f297cbcccb: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:36.282341 14968 tablet_bootstrap.cc:492] T b44efea240144200a816936377674b72 P d3397185e1f3439b9658a61e0992de39: Bootstrap starting.
I20260809 04:43:36.284133 14968 tablet_bootstrap.cc:654] T b44efea240144200a816936377674b72 P d3397185e1f3439b9658a61e0992de39: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:36.285037 14967 tablet_bootstrap.cc:492] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8: Bootstrap starting.
I20260809 04:43:36.287230 14967 tablet_bootstrap.cc:654] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:36.289228 14991 ts_tablet_manager.cc:1905] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:43:36.291990 14992 ts_tablet_manager.cc:1905] T 471e79cbe15a480da83ece93f5934c7b P 1af1e9ee279b4d3c9da36585d1b98fa8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:43:36.302510 14969 tablet_bootstrap.cc:492] T b44efea240144200a816936377674b72 P 1c91046a5e034383963fa5f297cbcccb: No bootstrap required, opened a new log
I20260809 04:43:36.302801 14969 ts_tablet_manager.cc:1397] T b44efea240144200a816936377674b72 P 1c91046a5e034383963fa5f297cbcccb: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.006s
I20260809 04:43:36.303773 14969 raft_consensus.cc:348] T b44efea240144200a816936377674b72 P 1c91046a5e034383963fa5f297cbcccb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } }
I20260809 04:43:36.304152 14969 raft_consensus.cc:374] T b44efea240144200a816936377674b72 P 1c91046a5e034383963fa5f297cbcccb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:36.304309 14969 raft_consensus.cc:729] T b44efea240144200a816936377674b72 P 1c91046a5e034383963fa5f297cbcccb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c91046a5e034383963fa5f297cbcccb, State: Initialized, Role: FOLLOWER
I20260809 04:43:36.304675 14969 consensus_queue.cc:260] T b44efea240144200a816936377674b72 P 1c91046a5e034383963fa5f297cbcccb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } }
I20260809 04:43:36.306121 14969 ts_tablet_manager.cc:1428] T b44efea240144200a816936377674b72 P 1c91046a5e034383963fa5f297cbcccb: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 04:43:36.310062 14992 ts_tablet_manager.cc:1918] T 471e79cbe15a480da83ece93f5934c7b P 1af1e9ee279b4d3c9da36585d1b98fa8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:43:36.310293 14992 log.cc:1196] T 471e79cbe15a480da83ece93f5934c7b P 1af1e9ee279b4d3c9da36585d1b98fa8: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-1/wal/wals/471e79cbe15a480da83ece93f5934c7b
I20260809 04:43:36.310945 14992 ts_tablet_manager.cc:1939] T 471e79cbe15a480da83ece93f5934c7b P 1af1e9ee279b4d3c9da36585d1b98fa8: Deleting consensus metadata
I20260809 04:43:36.312768 14967 tablet_bootstrap.cc:492] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8: No bootstrap required, opened a new log
I20260809 04:43:36.312943 14967 ts_tablet_manager.cc:1397] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8: Time spent bootstrapping tablet: real 0.028s user 0.009s sys 0.000s
I20260809 04:43:36.314050 14967 raft_consensus.cc:348] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } }
I20260809 04:43:36.314360 14967 raft_consensus.cc:374] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:36.314532 14967 raft_consensus.cc:729] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1af1e9ee279b4d3c9da36585d1b98fa8, State: Initialized, Role: FOLLOWER
I20260809 04:43:36.314909 14967 consensus_queue.cc:260] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } }
I20260809 04:43:36.316231 14991 ts_tablet_manager.cc:1918] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:43:36.316406 14991 log.cc:1196] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-0/wal/wals/471e79cbe15a480da83ece93f5934c7b
I20260809 04:43:36.316999 14991 ts_tablet_manager.cc:1939] T 471e79cbe15a480da83ece93f5934c7b P 1c91046a5e034383963fa5f297cbcccb: Deleting consensus metadata
I20260809 04:43:36.317984 14408 catalog_manager.cc:4841] TS 1af1e9ee279b4d3c9da36585d1b98fa8 (127.12.234.2:38947): tablet 471e79cbe15a480da83ece93f5934c7b (table test-table [id=6d0e31559b7a433fbf65f1df26fa7a1e]) successfully deleted
I20260809 04:43:36.318969 14967 ts_tablet_manager.cc:1428] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260809 04:43:36.320048 14993 tablet_replica.cc:330] T 471e79cbe15a480da83ece93f5934c7b P d3397185e1f3439b9658a61e0992de39: stopping tablet replica
I20260809 04:43:36.320715 14968 tablet_bootstrap.cc:492] T b44efea240144200a816936377674b72 P d3397185e1f3439b9658a61e0992de39: No bootstrap required, opened a new log
I20260809 04:43:36.321722 14968 ts_tablet_manager.cc:1397] T b44efea240144200a816936377674b72 P d3397185e1f3439b9658a61e0992de39: Time spent bootstrapping tablet: real 0.039s user 0.001s sys 0.019s
I20260809 04:43:36.321796 14408 catalog_manager.cc:4841] TS 1c91046a5e034383963fa5f297cbcccb (127.12.234.1:32795): tablet 471e79cbe15a480da83ece93f5934c7b (table test-table [id=6d0e31559b7a433fbf65f1df26fa7a1e]) successfully deleted
I20260809 04:43:36.322034 14993 raft_consensus.cc:2229] T 471e79cbe15a480da83ece93f5934c7b P d3397185e1f3439b9658a61e0992de39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:43:36.322538 14993 raft_consensus.cc:2258] T 471e79cbe15a480da83ece93f5934c7b P d3397185e1f3439b9658a61e0992de39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:43:36.323081 14968 raft_consensus.cc:348] T b44efea240144200a816936377674b72 P d3397185e1f3439b9658a61e0992de39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } }
I20260809 04:43:36.323349 14968 raft_consensus.cc:374] T b44efea240144200a816936377674b72 P d3397185e1f3439b9658a61e0992de39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:36.323685 14968 raft_consensus.cc:729] T b44efea240144200a816936377674b72 P d3397185e1f3439b9658a61e0992de39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3397185e1f3439b9658a61e0992de39, State: Initialized, Role: FOLLOWER
I20260809 04:43:36.324026 14968 consensus_queue.cc:260] T b44efea240144200a816936377674b72 P d3397185e1f3439b9658a61e0992de39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } }
I20260809 04:43:36.324967 14993 ts_tablet_manager.cc:1905] T 471e79cbe15a480da83ece93f5934c7b P d3397185e1f3439b9658a61e0992de39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:43:36.325090 14968 ts_tablet_manager.cc:1428] T b44efea240144200a816936377674b72 P d3397185e1f3439b9658a61e0992de39: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:43:36.332186 14993 ts_tablet_manager.cc:1918] T 471e79cbe15a480da83ece93f5934c7b P d3397185e1f3439b9658a61e0992de39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:43:36.332360 14993 log.cc:1196] T 471e79cbe15a480da83ece93f5934c7b P d3397185e1f3439b9658a61e0992de39: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-2/wal/wals/471e79cbe15a480da83ece93f5934c7b
I20260809 04:43:36.332890 14993 ts_tablet_manager.cc:1939] T 471e79cbe15a480da83ece93f5934c7b P d3397185e1f3439b9658a61e0992de39: Deleting consensus metadata
I20260809 04:43:36.334717 14406 catalog_manager.cc:4841] TS d3397185e1f3439b9658a61e0992de39 (127.12.234.3:37625): tablet 471e79cbe15a480da83ece93f5934c7b (table test-table [id=6d0e31559b7a433fbf65f1df26fa7a1e]) successfully deleted
I20260809 04:43:36.493995 14973 raft_consensus.cc:482] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:43:36.494251 14973 raft_consensus.cc:504] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } }
I20260809 04:43:36.496251 14973 leader_election.cc:288] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c91046a5e034383963fa5f297cbcccb (127.12.234.1:32795), d3397185e1f3439b9658a61e0992de39 (127.12.234.3:37625)
I20260809 04:43:36.510613 14964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:36.497412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:38245 (local address 127.12.234.3:37625)
0809 04:43:36.497655 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:36.497664 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:36.503150 (+ 5486us) server_negotiation.cc:407] Connection header received
0809 04:43:36.503464 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:36.503472 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:36.503612 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:36.503715 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:36.504646 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.505630 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.508345 (+ 2715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.509570 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.509749 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:36.509821 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:36.510199 (+ 378us) server_negotiation.cc:299] Negotiation successful
0809 04:43:36.510324 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 04:43:36.510771 14997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:36.496925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:37625 (local address 127.12.234.2:38245)
0809 04:43:36.503024 (+ 6099us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:36.503047 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:36.503277 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:36.503946 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:36.503951 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:36.503979 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:36.504399 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:36.504413 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.505754 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.505762 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:36.508139 (+ 2377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:36.508158 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.509814 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.509823 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:36.510027 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:36.510052 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 04:43:36.510161 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 04:43:36.510301 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5928,"mutex_wait_us":121,"thread_start_us":5758,"threads_started":1}
I20260809 04:43:36.511358 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:36.497337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:43923 (local address 127.12.234.1:32795)
0809 04:43:36.497534 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:36.497540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:36.503963 (+ 6423us) server_negotiation.cc:407] Connection header received
0809 04:43:36.504236 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:36.504240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:36.504297 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:36.504398 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:36.505113 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.506479 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.508817 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.509745 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.509898 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:36.509974 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:36.510445 (+ 471us) server_negotiation.cc:299] Negotiation successful
0809 04:43:36.510576 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 04:43:36.511467 14998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:36.497102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:32795 (local address 127.12.234.2:43923)
0809 04:43:36.503851 (+ 6749us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:36.503870 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:36.504056 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:36.504417 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:36.504421 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:36.504449 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:36.504908 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:36.504921 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.506632 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.506638 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:36.508667 (+ 2029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:36.508678 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.510006 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.510013 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:36.510225 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:36.510262 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 04:43:36.510961 (+ 699us) client_negotiation.cc:240] Negotiation successful
0809 04:43:36.511094 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6621,"mutex_wait_us":3,"spinlock_wait_cycles":28416,"thread_start_us":6356,"threads_started":1}
I20260809 04:43:36.511484 14899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b44efea240144200a816936377674b72" candidate_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3397185e1f3439b9658a61e0992de39" is_pre_election: true
I20260809 04:43:36.511761 14899 raft_consensus.cc:2454] T b44efea240144200a816936377674b72 P d3397185e1f3439b9658a61e0992de39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1af1e9ee279b4d3c9da36585d1b98fa8 in term 0.
I20260809 04:43:36.512319 14671 leader_election.cc:302] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1af1e9ee279b4d3c9da36585d1b98fa8, d3397185e1f3439b9658a61e0992de39; no voters:
I20260809 04:43:36.512742 14973 raft_consensus.cc:2781] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:43:36.512874 14973 raft_consensus.cc:482] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:43:36.512719 14604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b44efea240144200a816936377674b72" candidate_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c91046a5e034383963fa5f297cbcccb" is_pre_election: true
I20260809 04:43:36.513002 14973 raft_consensus.cc:3037] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:36.513199 14604 raft_consensus.cc:2454] T b44efea240144200a816936377674b72 P 1c91046a5e034383963fa5f297cbcccb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1af1e9ee279b4d3c9da36585d1b98fa8 in term 0.
I20260809 04:43:36.515611 14973 raft_consensus.cc:504] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } }
I20260809 04:43:36.516399 14973 leader_election.cc:288] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8 [CANDIDATE]: Term 1 election: Requested vote from peers 1c91046a5e034383963fa5f297cbcccb (127.12.234.1:32795), d3397185e1f3439b9658a61e0992de39 (127.12.234.3:37625)
I20260809 04:43:36.516808 14604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b44efea240144200a816936377674b72" candidate_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c91046a5e034383963fa5f297cbcccb"
I20260809 04:43:36.517022 14604 raft_consensus.cc:3037] T b44efea240144200a816936377674b72 P 1c91046a5e034383963fa5f297cbcccb [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:36.517114 14899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b44efea240144200a816936377674b72" candidate_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3397185e1f3439b9658a61e0992de39"
I20260809 04:43:36.517335 14899 raft_consensus.cc:3037] T b44efea240144200a816936377674b72 P d3397185e1f3439b9658a61e0992de39 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:36.519353 14899 raft_consensus.cc:2454] T b44efea240144200a816936377674b72 P d3397185e1f3439b9658a61e0992de39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1af1e9ee279b4d3c9da36585d1b98fa8 in term 1.
I20260809 04:43:36.519744 14671 leader_election.cc:302] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1af1e9ee279b4d3c9da36585d1b98fa8, d3397185e1f3439b9658a61e0992de39; no voters:
I20260809 04:43:36.520042 14973 raft_consensus.cc:2781] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:43:36.520656 14973 raft_consensus.cc:686] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 1 LEADER]: Becoming Leader. State: Replica: 1af1e9ee279b4d3c9da36585d1b98fa8, State: Running, Role: LEADER
I20260809 04:43:36.520759 14604 raft_consensus.cc:2454] T b44efea240144200a816936377674b72 P 1c91046a5e034383963fa5f297cbcccb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1af1e9ee279b4d3c9da36585d1b98fa8 in term 1.
I20260809 04:43:36.521112 14973 consensus_queue.cc:237] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } }
I20260809 04:43:36.524729 14421 catalog_manager.cc:5495] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1af1e9ee279b4d3c9da36585d1b98fa8 (127.12.234.2). New cstate: current_term: 1 leader_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } health_report { overall_health: UNKNOWN } } }
Aug 09 04:43:36 dist-test-slave-1jjb krb5kdc[14376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250613, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:36.566654 15006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:36.550775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40905 (local address 127.0.0.1:53868)
0809 04:43:36.551641 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:36.551687 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:36.551851 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:36.552323 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:36.552326 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:36.552993 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:36.553203 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:36.553212 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.554734 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.554738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:36.555720 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:36.555731 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.556822 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.556826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:36.557023 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:36.557752 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:36.557773 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:36.560280 (+ 2507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:36.564200 (+ 3920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:36.564210 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:36.564216 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:36.564642 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:36.565046 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:36.565049 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:36.565051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:36.565179 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:36.565826 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:36.565831 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:36.565975 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 04:43:36.566111 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 04:43:36.566310 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"mutex_wait_us":21,"thread_start_us":434,"threads_started":1}
I20260809 04:43:36.567003 15008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:36.550822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53868 (local address 127.12.234.62:40905)
0809 04:43:36.551964 (+ 1142us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:36.551970 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:36.552004 (+ 34us) server_negotiation.cc:407] Connection header received
0809 04:43:36.552066 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:36.552097 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:36.552175 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:36.552294 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:36.553392 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.554589 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.555878 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.556642 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.556805 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:36.560482 (+ 3677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:36.560489 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:36.560494 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:36.560532 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:36.564040 (+ 3508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:36.564767 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:36.564772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:36.564776 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:36.564932 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:36.565300 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:36.565305 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:36.565308 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:36.565694 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:36.565850 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:36.566235 (+ 385us) server_negotiation.cc:299] Negotiation successful
0809 04:43:36.566482 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":1005,"thread_start_us":845,"threads_started":1}
I20260809 04:43:36.573068 14421 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:53868:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 04:43:36.574385 14421 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:43:36.581490 14578 tablet_service.cc:1467] Processing CreateTablet for tablet 5ce4ff203662435dba8217d29431f7c3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e4f2382bf7f94621906c54d7156416ef]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:43:36.581736 14726 tablet_service.cc:1467] Processing CreateTablet for tablet 5ce4ff203662435dba8217d29431f7c3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e4f2382bf7f94621906c54d7156416ef]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:43:36.582227 14578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ce4ff203662435dba8217d29431f7c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:36.582136 14872 tablet_service.cc:1467] Processing CreateTablet for tablet 5ce4ff203662435dba8217d29431f7c3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e4f2382bf7f94621906c54d7156416ef]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:43:36.582376 14726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ce4ff203662435dba8217d29431f7c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:36.582751 14872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ce4ff203662435dba8217d29431f7c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:36.590912 14969 tablet_bootstrap.cc:492] T 5ce4ff203662435dba8217d29431f7c3 P 1c91046a5e034383963fa5f297cbcccb: Bootstrap starting.
I20260809 04:43:36.591370 14967 tablet_bootstrap.cc:492] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8: Bootstrap starting.
I20260809 04:43:36.593036 14969 tablet_bootstrap.cc:654] T 5ce4ff203662435dba8217d29431f7c3 P 1c91046a5e034383963fa5f297cbcccb: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:36.593231 14967 tablet_bootstrap.cc:654] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:36.594506 14968 tablet_bootstrap.cc:492] T 5ce4ff203662435dba8217d29431f7c3 P d3397185e1f3439b9658a61e0992de39: Bootstrap starting.
I20260809 04:43:36.597206 14968 tablet_bootstrap.cc:654] T 5ce4ff203662435dba8217d29431f7c3 P d3397185e1f3439b9658a61e0992de39: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:36.604187 14967 tablet_bootstrap.cc:492] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8: No bootstrap required, opened a new log
I20260809 04:43:36.604454 14967 ts_tablet_manager.cc:1397] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.008s
I20260809 04:43:36.605566 14967 raft_consensus.cc:348] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } }
I20260809 04:43:36.605868 14967 raft_consensus.cc:374] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:36.605978 14967 raft_consensus.cc:729] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1af1e9ee279b4d3c9da36585d1b98fa8, State: Initialized, Role: FOLLOWER
I20260809 04:43:36.606359 14967 consensus_queue.cc:260] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } }
I20260809 04:43:36.606913 14973 tablet_replica.cc:439] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } } }
I20260809 04:43:36.607250 14973 tablet_replica.cc:442] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:43:36.608671 14968 tablet_bootstrap.cc:492] T 5ce4ff203662435dba8217d29431f7c3 P d3397185e1f3439b9658a61e0992de39: No bootstrap required, opened a new log
I20260809 04:43:36.608927 14968 ts_tablet_manager.cc:1397] T 5ce4ff203662435dba8217d29431f7c3 P d3397185e1f3439b9658a61e0992de39: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.009s
I20260809 04:43:36.609794 14968 raft_consensus.cc:348] T 5ce4ff203662435dba8217d29431f7c3 P d3397185e1f3439b9658a61e0992de39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } }
I20260809 04:43:36.610064 14968 raft_consensus.cc:374] T 5ce4ff203662435dba8217d29431f7c3 P d3397185e1f3439b9658a61e0992de39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:36.610163 14968 raft_consensus.cc:729] T 5ce4ff203662435dba8217d29431f7c3 P d3397185e1f3439b9658a61e0992de39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3397185e1f3439b9658a61e0992de39, State: Initialized, Role: FOLLOWER
I20260809 04:43:36.610364 14967 ts_tablet_manager.cc:1428] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260809 04:43:36.610440 14968 consensus_queue.cc:260] T 5ce4ff203662435dba8217d29431f7c3 P d3397185e1f3439b9658a61e0992de39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } }
I20260809 04:43:36.610903 14975 tablet_replica.cc:439] T 5ce4ff203662435dba8217d29431f7c3 P d3397185e1f3439b9658a61e0992de39: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } } }
I20260809 04:43:36.611310 14975 tablet_replica.cc:442] T 5ce4ff203662435dba8217d29431f7c3 P d3397185e1f3439b9658a61e0992de39: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:43:36.611792 14968 ts_tablet_manager.cc:1428] T 5ce4ff203662435dba8217d29431f7c3 P d3397185e1f3439b9658a61e0992de39: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 04:43:36.617365 14969 tablet_bootstrap.cc:492] T 5ce4ff203662435dba8217d29431f7c3 P 1c91046a5e034383963fa5f297cbcccb: No bootstrap required, opened a new log
I20260809 04:43:36.617570 14969 ts_tablet_manager.cc:1397] T 5ce4ff203662435dba8217d29431f7c3 P 1c91046a5e034383963fa5f297cbcccb: Time spent bootstrapping tablet: real 0.027s user 0.015s sys 0.004s
I20260809 04:43:36.618405 14969 raft_consensus.cc:348] T 5ce4ff203662435dba8217d29431f7c3 P 1c91046a5e034383963fa5f297cbcccb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } }
I20260809 04:43:36.618693 14969 raft_consensus.cc:374] T 5ce4ff203662435dba8217d29431f7c3 P 1c91046a5e034383963fa5f297cbcccb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:36.618798 14969 raft_consensus.cc:729] T 5ce4ff203662435dba8217d29431f7c3 P 1c91046a5e034383963fa5f297cbcccb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c91046a5e034383963fa5f297cbcccb, State: Initialized, Role: FOLLOWER
I20260809 04:43:36.619066 14969 consensus_queue.cc:260] T 5ce4ff203662435dba8217d29431f7c3 P 1c91046a5e034383963fa5f297cbcccb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } }
I20260809 04:43:36.619680 14969 ts_tablet_manager.cc:1428] T 5ce4ff203662435dba8217d29431f7c3 P 1c91046a5e034383963fa5f297cbcccb: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 04:43:36.619573 14974 tablet_replica.cc:439] T 5ce4ff203662435dba8217d29431f7c3 P 1c91046a5e034383963fa5f297cbcccb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } } }
I20260809 04:43:36.619894 14974 tablet_replica.cc:442] T 5ce4ff203662435dba8217d29431f7c3 P 1c91046a5e034383963fa5f297cbcccb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:43:36.774001 14973 raft_consensus.cc:482] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:43:36.774267 14973 raft_consensus.cc:504] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } }
I20260809 04:43:36.775013 14973 leader_election.cc:288] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c91046a5e034383963fa5f297cbcccb (127.12.234.1:32795), d3397185e1f3439b9658a61e0992de39 (127.12.234.3:37625)
I20260809 04:43:36.775569 14604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce4ff203662435dba8217d29431f7c3" candidate_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c91046a5e034383963fa5f297cbcccb" is_pre_election: true
I20260809 04:43:36.775642 14899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce4ff203662435dba8217d29431f7c3" candidate_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3397185e1f3439b9658a61e0992de39" is_pre_election: true
I20260809 04:43:36.775849 14604 raft_consensus.cc:2454] T 5ce4ff203662435dba8217d29431f7c3 P 1c91046a5e034383963fa5f297cbcccb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1af1e9ee279b4d3c9da36585d1b98fa8 in term 0.
I20260809 04:43:36.775962 14899 raft_consensus.cc:2454] T 5ce4ff203662435dba8217d29431f7c3 P d3397185e1f3439b9658a61e0992de39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1af1e9ee279b4d3c9da36585d1b98fa8 in term 0.
I20260809 04:43:36.776355 14673 leader_election.cc:302] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1af1e9ee279b4d3c9da36585d1b98fa8, 1c91046a5e034383963fa5f297cbcccb; no voters:
I20260809 04:43:36.776678 14973 raft_consensus.cc:2781] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:43:36.776767 14973 raft_consensus.cc:482] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:43:36.776926 14973 raft_consensus.cc:3037] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:36.778977 14973 raft_consensus.cc:504] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } }
I20260809 04:43:36.779498 14973 leader_election.cc:288] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8 [CANDIDATE]: Term 1 election: Requested vote from peers 1c91046a5e034383963fa5f297cbcccb (127.12.234.1:32795), d3397185e1f3439b9658a61e0992de39 (127.12.234.3:37625)
I20260809 04:43:36.780022 14604 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce4ff203662435dba8217d29431f7c3" candidate_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c91046a5e034383963fa5f297cbcccb"
I20260809 04:43:36.780022 14899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ce4ff203662435dba8217d29431f7c3" candidate_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3397185e1f3439b9658a61e0992de39"
I20260809 04:43:36.780279 14604 raft_consensus.cc:3037] T 5ce4ff203662435dba8217d29431f7c3 P 1c91046a5e034383963fa5f297cbcccb [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:36.780288 14899 raft_consensus.cc:3037] T 5ce4ff203662435dba8217d29431f7c3 P d3397185e1f3439b9658a61e0992de39 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:36.783310 14604 raft_consensus.cc:2454] T 5ce4ff203662435dba8217d29431f7c3 P 1c91046a5e034383963fa5f297cbcccb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1af1e9ee279b4d3c9da36585d1b98fa8 in term 1.
I20260809 04:43:36.783311 14899 raft_consensus.cc:2454] T 5ce4ff203662435dba8217d29431f7c3 P d3397185e1f3439b9658a61e0992de39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1af1e9ee279b4d3c9da36585d1b98fa8 in term 1.
I20260809 04:43:36.783742 14673 leader_election.cc:302] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1af1e9ee279b4d3c9da36585d1b98fa8, 1c91046a5e034383963fa5f297cbcccb, d3397185e1f3439b9658a61e0992de39; no voters:
I20260809 04:43:36.784029 14973 raft_consensus.cc:2781] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:43:36.784220 14973 raft_consensus.cc:686] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 1 LEADER]: Becoming Leader. State: Replica: 1af1e9ee279b4d3c9da36585d1b98fa8, State: Running, Role: LEADER
I20260809 04:43:36.784484 14973 consensus_queue.cc:237] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } }
I20260809 04:43:36.786046 14999 tablet_replica.cc:439] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8: TxnStatusTablet state changed. Reason: New leader 1af1e9ee279b4d3c9da36585d1b98fa8. Latest consensus state: current_term: 1 leader_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } } }
I20260809 04:43:36.786393 14999 tablet_replica.cc:442] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:43:36.787003 15015 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 04:43:36.787213 15015 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 04:43:36.787328 14422 catalog_manager.cc:5495] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1af1e9ee279b4d3c9da36585d1b98fa8 (127.12.234.2). New cstate: current_term: 1 leader_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } health_report { overall_health: UNKNOWN } } }
I20260809 04:43:36.842314 15006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:36.835354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:38947 (local address 127.0.0.1:42840)
0809 04:43:36.835590 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:36.835617 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:36.835797 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:36.836525 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:36.836529 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:36.836538 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:43:36.836717 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:36.836725 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.838378 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.838382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:36.839371 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:36.839383 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.840388 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.840392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:36.840533 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:36.840589 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:43:36.841745 (+ 1156us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:43:36.841765 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 04:43:36.841844 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 04:43:36.842076 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 04:43:36.842379 15022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:36.835496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42840 (local address 127.12.234.2:38947)
0809 04:43:36.836053 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:36.836060 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:36.836110 (+ 50us) server_negotiation.cc:407] Connection header received
0809 04:43:36.836217 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:36.836223 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:36.836324 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:36.836499 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:43:36.836915 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.838227 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.839519 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.840244 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.840371 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:36.840783 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:43:36.841596 (+ 813us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:43:36.841755 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:36.841919 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 04:43:36.842006 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":270,"threads_started":1}
I20260809 04:43:36.847707 14899 raft_consensus.cc:1261] T 5ce4ff203662435dba8217d29431f7c3 P d3397185e1f3439b9658a61e0992de39 [term 1 FOLLOWER]: Refusing update from remote peer 1af1e9ee279b4d3c9da36585d1b98fa8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:43:36.847864 14604 raft_consensus.cc:1261] T 5ce4ff203662435dba8217d29431f7c3 P 1c91046a5e034383963fa5f297cbcccb [term 1 FOLLOWER]: Refusing update from remote peer 1af1e9ee279b4d3c9da36585d1b98fa8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:43:36.848510 14999 consensus_queue.cc:1035] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:43:36.848905 14973 consensus_queue.cc:1035] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:43:36.852244 14975 tablet_replica.cc:439] T 5ce4ff203662435dba8217d29431f7c3 P d3397185e1f3439b9658a61e0992de39: TxnStatusTablet state changed. Reason: New leader 1af1e9ee279b4d3c9da36585d1b98fa8. Latest consensus state: current_term: 1 leader_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } } }
I20260809 04:43:36.852546 14975 tablet_replica.cc:442] T 5ce4ff203662435dba8217d29431f7c3 P d3397185e1f3439b9658a61e0992de39: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:43:36.859375 15023 mvcc.cc:205] Tried to move back new op lower bound from 7316482526599786496 to 7316482526351368192. Current Snapshot: MvccSnapshot[applied={T|T < 7316482526599786496}]
I20260809 04:43:36.864791 14975 tablet_replica.cc:439] T 5ce4ff203662435dba8217d29431f7c3 P d3397185e1f3439b9658a61e0992de39: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } } }
I20260809 04:43:36.865119 14975 tablet_replica.cc:442] T 5ce4ff203662435dba8217d29431f7c3 P d3397185e1f3439b9658a61e0992de39: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:43:36.867967 14974 tablet_replica.cc:439] T 5ce4ff203662435dba8217d29431f7c3 P 1c91046a5e034383963fa5f297cbcccb: TxnStatusTablet state changed. Reason: New leader 1af1e9ee279b4d3c9da36585d1b98fa8. Latest consensus state: current_term: 1 leader_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } } }
I20260809 04:43:36.868259 14974 tablet_replica.cc:442] T 5ce4ff203662435dba8217d29431f7c3 P 1c91046a5e034383963fa5f297cbcccb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:43:36.868530 14973 tablet_replica.cc:439] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } } }
I20260809 04:43:36.868876 14973 tablet_replica.cc:442] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:43:36.872401 14999 tablet_replica.cc:439] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } } }
I20260809 04:43:36.872813 14999 tablet_replica.cc:442] T 5ce4ff203662435dba8217d29431f7c3 P 1af1e9ee279b4d3c9da36585d1b98fa8: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:43:36.873533 14974 tablet_replica.cc:439] T 5ce4ff203662435dba8217d29431f7c3 P 1c91046a5e034383963fa5f297cbcccb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 } } peers { permanent_uuid: "1af1e9ee279b4d3c9da36585d1b98fa8" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38947 } } peers { permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 } } }
I20260809 04:43:36.873920 14974 tablet_replica.cc:442] T 5ce4ff203662435dba8217d29431f7c3 P 1c91046a5e034383963fa5f297cbcccb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:43:36.882863 15022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:36.876716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42842 (local address 127.12.234.2:38947)
0809 04:43:36.876889 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:36.876896 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:36.877034 (+ 138us) server_negotiation.cc:407] Connection header received
0809 04:43:36.877320 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:36.877325 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:36.877414 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:36.877513 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:43:36.878074 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.879282 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.880804 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.881517 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.881655 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:36.882013 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:43:36.882142 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:43:36.882235 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:36.882382 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 04:43:36.882462 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:43:36.900935 15035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:36.893923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:38947 (local address 127.12.234.2:42323)
0809 04:43:36.894619 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:36.894634 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:36.894754 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:36.895136 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:36.895139 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:36.895148 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:43:36.895584 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:36.895593 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.897248 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.897260 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:36.898555 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:36.898567 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.899641 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.899646 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:36.899781 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:36.899840 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:43:36.900292 (+ 452us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:43:36.900302 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 04:43:36.900368 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 04:43:36.900471 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":355,"threads_started":1}
I20260809 04:43:36.900967 15022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:36.893986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:42323 (local address 127.12.234.2:38947)
0809 04:43:36.894142 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:36.894146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:36.894717 (+ 571us) server_negotiation.cc:407] Connection header received
0809 04:43:36.894905 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:36.894916 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:36.895016 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:36.895107 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:43:36.895737 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.897003 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.898688 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:36.899463 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:36.899605 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:36.900026 (+ 421us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:43:36.900196 (+ 170us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:43:36.900292 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:36.900416 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 04:43:36.900496 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 04:43:36.910945 14999 consensus_queue.cc:1035] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c91046a5e034383963fa5f297cbcccb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:43:36.912292 14899 raft_consensus.cc:1261] T b44efea240144200a816936377674b72 P d3397185e1f3439b9658a61e0992de39 [term 1 FOLLOWER]: Refusing update from remote peer 1af1e9ee279b4d3c9da36585d1b98fa8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:43:36.913298 14999 consensus_queue.cc:1035] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3397185e1f3439b9658a61e0992de39" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 37625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:43:36.921046 15024 mvcc.cc:205] Tried to move back new op lower bound from 7316482526866288640 to 7316482525274521600. Current Snapshot: MvccSnapshot[applied={T|T < 7316482526866288640}]
I20260809 04:43:36.923182 15025 mvcc.cc:205] Tried to move back new op lower bound from 7316482526866288640 to 7316482525274521600. Current Snapshot: MvccSnapshot[applied={T|T < 7316482526866288640}]
I20260809 04:43:36.982481 14421 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:43:36.982793 14421 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:43:36.985901 14421 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ae5eda3671af44229eb9257c969e2976: Sending DeleteTablet for 3 replicas of tablet b44efea240144200a816936377674b72
I20260809 04:43:36.986816 14578 tablet_service.cc:1514] Processing DeleteTablet for tablet b44efea240144200a816936377674b72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:43:36 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:60202
I20260809 04:43:36.986910 14872 tablet_service.cc:1514] Processing DeleteTablet for tablet b44efea240144200a816936377674b72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:43:36 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:45352
I20260809 04:43:36.994016 14726 tablet_service.cc:1514] Processing DeleteTablet for tablet b44efea240144200a816936377674b72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:43:36 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:42838
I20260809 04:43:36.997259 15045 tablet_replica.cc:330] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8: stopping tablet replica
I20260809 04:43:36.998551 15045 raft_consensus.cc:2229] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:43:36.999637 15043 tablet_replica.cc:330] T b44efea240144200a816936377674b72 P d3397185e1f3439b9658a61e0992de39: stopping tablet replica
I20260809 04:43:36.999886 15043 raft_consensus.cc:2229] T b44efea240144200a816936377674b72 P d3397185e1f3439b9658a61e0992de39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:43:37.000272 15043 raft_consensus.cc:2258] T b44efea240144200a816936377674b72 P d3397185e1f3439b9658a61e0992de39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:43:37.001093 15042 tablet_replica.cc:330] T b44efea240144200a816936377674b72 P 1c91046a5e034383963fa5f297cbcccb: stopping tablet replica
I20260809 04:43:37.001365 15042 raft_consensus.cc:2229] T b44efea240144200a816936377674b72 P 1c91046a5e034383963fa5f297cbcccb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:43:37.001652 15043 ts_tablet_manager.cc:1905] T b44efea240144200a816936377674b72 P d3397185e1f3439b9658a61e0992de39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:43:37.001699 15042 raft_consensus.cc:2258] T b44efea240144200a816936377674b72 P 1c91046a5e034383963fa5f297cbcccb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:43:37.003042 15042 ts_tablet_manager.cc:1905] T b44efea240144200a816936377674b72 P 1c91046a5e034383963fa5f297cbcccb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:43:37.008278 15045 raft_consensus.cc:2258] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:43:37.009591 15045 ts_tablet_manager.cc:1905] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:43:37.010524 15042 ts_tablet_manager.cc:1918] T b44efea240144200a816936377674b72 P 1c91046a5e034383963fa5f297cbcccb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:43:37.010715 15042 log.cc:1196] T b44efea240144200a816936377674b72 P 1c91046a5e034383963fa5f297cbcccb: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-0/wal/wals/b44efea240144200a816936377674b72
I20260809 04:43:37.011251 15042 ts_tablet_manager.cc:1939] T b44efea240144200a816936377674b72 P 1c91046a5e034383963fa5f297cbcccb: Deleting consensus metadata
I20260809 04:43:37.012557 14408 catalog_manager.cc:4841] TS 1c91046a5e034383963fa5f297cbcccb (127.12.234.1:32795): tablet b44efea240144200a816936377674b72 (table test-table [id=b63ccc48b1504e72a3f19311b5760322]) successfully deleted
I20260809 04:43:37.013779 15043 ts_tablet_manager.cc:1918] T b44efea240144200a816936377674b72 P d3397185e1f3439b9658a61e0992de39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:43:37.013991 15043 log.cc:1196] T b44efea240144200a816936377674b72 P d3397185e1f3439b9658a61e0992de39: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-2/wal/wals/b44efea240144200a816936377674b72
I20260809 04:43:37.014640 15043 ts_tablet_manager.cc:1939] T b44efea240144200a816936377674b72 P d3397185e1f3439b9658a61e0992de39: Deleting consensus metadata
I20260809 04:43:37.015954 14406 catalog_manager.cc:4841] TS d3397185e1f3439b9658a61e0992de39 (127.12.234.3:37625): tablet b44efea240144200a816936377674b72 (table test-table [id=b63ccc48b1504e72a3f19311b5760322]) successfully deleted
I20260809 04:43:37.017367 15045 ts_tablet_manager.cc:1918] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:43:37.017539 15045 log.cc:1196] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786250603326224-13224-0/minicluster-data/ts-1/wal/wals/b44efea240144200a816936377674b72
I20260809 04:43:37.018098 15045 ts_tablet_manager.cc:1939] T b44efea240144200a816936377674b72 P 1af1e9ee279b4d3c9da36585d1b98fa8: Deleting consensus metadata
I20260809 04:43:37.019351 14408 catalog_manager.cc:4841] TS 1af1e9ee279b4d3c9da36585d1b98fa8 (127.12.234.2:38947): tablet b44efea240144200a816936377674b72 (table test-table [id=b63ccc48b1504e72a3f19311b5760322]) successfully deleted
Aug 09 04:43:37 dist-test-slave-1jjb krb5kdc[14376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250615, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.234.1@KRBTEST.COM
I20260809 04:43:37.033262 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:37.011500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60216 (local address 127.12.234.1:32795)
0809 04:43:37.011694 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:37.011700 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:37.017097 (+ 5397us) server_negotiation.cc:407] Connection header received
0809 04:43:37.017440 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:37.017448 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:37.017559 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:37.017643 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:37.020132 (+ 2489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:37.020856 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:37.022183 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:37.022613 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:37.022750 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:37.027024 (+ 4274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:37.027037 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:37.027055 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:37.027095 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:37.030193 (+ 3098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:37.030945 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:37.030958 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:37.030963 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:37.031153 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:37.031555 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:37.031561 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:37.031564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:37.031933 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:37.032262 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:37.032648 (+ 386us) server_negotiation.cc:299] Negotiation successful
0809 04:43:37.032897 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 04:43:37.034171 14645 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60216
I20260809 04:43:37.056032 15008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:37.041348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53878 (local address 127.12.234.62:40905)
0809 04:43:37.041723 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:37.041730 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:37.042158 (+ 428us) server_negotiation.cc:407] Connection header received
0809 04:43:37.042321 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:37.042328 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:37.042409 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:37.042524 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:37.043792 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:37.044950 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:37.046272 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:37.047004 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:37.047128 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:37.049736 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:37.049742 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:37.049746 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:37.049787 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:37.053423 (+ 3636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:37.054033 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:37.054039 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:37.054042 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:37.054184 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:37.054528 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:37.054533 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:37.054537 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:37.054918 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:37.055162 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:37.055441 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 04:43:37.055684 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
W20260809 04:43:37.056771 14421 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53878
I20260809 04:43:37.060403 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 14507
I20260809 04:43:37.073949 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:37.061137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60224 (local address 127.12.234.1:32795)
0809 04:43:37.061347 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:37.061354 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:37.061649 (+ 295us) server_negotiation.cc:407] Connection header received
0809 04:43:37.061840 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:37.061845 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:37.061916 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:37.062015 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:37.063108 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:37.063906 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:37.065177 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:37.065602 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:37.065738 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:37.067985 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:37.067990 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:37.067993 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:37.068025 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:37.071212 (+ 3187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:37.071996 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:37.072000 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:37.072002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:37.072192 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:37.072607 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:37.072611 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:37.072613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:37.072903 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:37.073108 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:37.073483 (+ 375us) server_negotiation.cc:299] Negotiation successful
0809 04:43:37.073652 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260809 04:43:37.074570 14645 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60224
W20260809 04:43:37.075140 13224 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 14507: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:43:37.075331 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 14507
I20260809 04:43:37.102895 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 14654
Aug 09 04:43:37 dist-test-slave-1jjb krb5kdc[14376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250615, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.234.2@KRBTEST.COM
I20260809 04:43:37.121551 15022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:37.103793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42844 (local address 127.12.234.2:38947)
0809 04:43:37.103988 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:37.103992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:37.104010 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:43:37.104203 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:37.104207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:37.104284 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:37.104378 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:37.105893 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:37.106619 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:37.108457 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:37.108947 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:37.109049 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:37.114429 (+ 5380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:37.114446 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:37.114473 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:37.114515 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:37.117963 (+ 3448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:37.118892 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:37.118900 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:37.118903 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:37.119055 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:37.119507 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:37.119512 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:37.119514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:37.120063 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:37.120600 (+ 537us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:37.120976 (+ 376us) server_negotiation.cc:299] Negotiation successful
0809 04:43:37.121178 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260809 04:43:37.122331 14795 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42844
W20260809 04:43:37.123009 13224 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 14654: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:43:37.123190 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 14654
I20260809 04:43:37.149397 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 14805
Aug 09 04:43:37 dist-test-slave-1jjb krb5kdc[14376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250615, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.234.3@KRBTEST.COM
I20260809 04:43:37.165352 15050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:37.150225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45362 (local address 127.12.234.3:37625)
0809 04:43:37.150738 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:37.150743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:37.150760 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:43:37.150816 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:37.150820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:37.150879 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:37.150972 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:37.152297 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:37.153018 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:37.154272 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:37.154722 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:37.154818 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:37.159089 (+ 4271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:37.159107 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:37.159128 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:37.159172 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:37.162421 (+ 3249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:37.163144 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:37.163152 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:37.163154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:37.163292 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:37.163667 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:37.163670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:37.163672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:37.164051 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:37.164371 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:37.164838 (+ 467us) server_negotiation.cc:299] Negotiation successful
0809 04:43:37.165024 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":295,"threads_started":1}
W20260809 04:43:37.166293 14943 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45362
W20260809 04:43:37.167011 13224 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 14805: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:43:37.167199 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 14805
I20260809 04:43:37.190932 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 14389
I20260809 04:43:37.192212 14466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:43:37.307837 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 14389
2026-08-09T04:43:37Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5084 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:43:37 dist-test-slave-1jjb krb5kdc[15054](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:43:37 dist-test-slave-1jjb krb5kdc[15054](info): set up 2 sockets
Aug 09 04:43:37 dist-test-slave-1jjb krb5kdc[15054](info): commencing operation
krb5kdc: starting...
W20260809 04:43:38.487929 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:43:38 dist-test-slave-1jjb krb5kdc[15054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:43:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:43:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:43:38.657850 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:42671
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:42285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:42671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:38.954686 15068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:38.955011 15068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:38.955178 15068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:38.962852 15068 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:43:38.962980 15068 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 04:43:38.963027 15068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:38.963145 15068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:38.963214 15068 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:43:38.963312 15068 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:43:38.974059 15068 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:42285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:42671
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:42671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:38.975610 15068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:38.977205 15068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:38.992165 15076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:38.992133 15074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:38.992529 15073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:38.992895 15068 server_base.cc:1034] running on GCE node
I20260809 04:43:38.993835 15068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:38.995472 15068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:38.996781 15068 hybrid_clock.cc:648] HybridClock initialized: now 1786250618996692 us; error 69 us; skew 500 ppm
I20260809 04:43:38.997179 15068 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:38 dist-test-slave-1jjb krb5kdc[15054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250618, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:39.001371 15068 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:43:39.003782 15068 webserver.cc:460] Webserver started at http://127.12.234.62:37683/ using document root <none> and password file <none>
I20260809 04:43:39.004881 15068 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:39.005035 15068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:39.005457 15068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:39.010010 15068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "e2763f4b36424b518849a07d07a5f916"
format_stamp: "Formatted at 2026-08-09 04:43:39 on dist-test-slave-1jjb"
server_key: "fd590331fa94e0e9db61b2153bc614b1"
server_key_iv: "0d84acee318cfa8880406488a4abc6d6"
server_key_version: "encryptionkey@0"
I20260809 04:43:39.011137 15068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "e2763f4b36424b518849a07d07a5f916"
format_stamp: "Formatted at 2026-08-09 04:43:39 on dist-test-slave-1jjb"
server_key: "fd590331fa94e0e9db61b2153bc614b1"
server_key_iv: "0d84acee318cfa8880406488a4abc6d6"
server_key_version: "encryptionkey@0"
I20260809 04:43:39.017735 15068 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.001s
I20260809 04:43:39.022101 15083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:39.023949 15068 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260809 04:43:39.024266 15068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "e2763f4b36424b518849a07d07a5f916"
format_stamp: "Formatted at 2026-08-09 04:43:39 on dist-test-slave-1jjb"
server_key: "fd590331fa94e0e9db61b2153bc614b1"
server_key_iv: "0d84acee318cfa8880406488a4abc6d6"
server_key_version: "encryptionkey@0"
I20260809 04:43:39.024597 15068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:39.059123 15068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:39.064004 15068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:39.064371 15068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:39.091293 15068 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:42671
I20260809 04:43:39.091316 15145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:42671 every 8 connection(s)
I20260809 04:43:39.092981 15068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:43:39.095604 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 15068
I20260809 04:43:39.095904 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:43:39.096771 13224 external_mini_cluster.cc:1428] Setting key d773291bd0becac3f14b983f11ec3e9b
I20260809 04:43:39.099262 15146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 04:43:39 dist-test-slave-1jjb krb5kdc[15054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:39.115903 15146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2763f4b36424b518849a07d07a5f916: Bootstrap starting.
I20260809 04:43:39.120747 15146 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2763f4b36424b518849a07d07a5f916: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:39.122488 15146 log.cc:824] T 00000000000000000000000000000000 P e2763f4b36424b518849a07d07a5f916: Log is configured to *not* fsync() on all Append() calls
I20260809 04:43:39.127454 15146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2763f4b36424b518849a07d07a5f916: No bootstrap required, opened a new log
I20260809 04:43:39.129387 15149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:39.099703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38628 (local address 127.12.234.62:42671)
0809 04:43:39.100792 (+ 1089us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:39.100817 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:39.100899 (+ 82us) server_negotiation.cc:407] Connection header received
0809 04:43:39.103583 (+ 2684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:39.103655 (+ 72us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:39.104434 (+ 779us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:39.105056 (+ 622us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:39.106659 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:39.109178 (+ 2519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:39.111146 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:39.112415 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:39.112679 (+ 264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:39.117232 (+ 4553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:39.117246 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:39.117270 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:39.117339 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:39.122206 (+ 4867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:39.124194 (+ 1988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:39.124205 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:39.124213 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:39.124499 (+ 286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:39.125092 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:39.125097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:39.125100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:39.125938 (+ 838us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:39.126346 (+ 408us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:39.127018 (+ 672us) server_negotiation.cc:299] Negotiation successful
0809 04:43:39.127608 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":407,"threads_started":1}
I20260809 04:43:39.137737 15146 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2763f4b36424b518849a07d07a5f916 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2763f4b36424b518849a07d07a5f916" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42671 } }
I20260809 04:43:39.138168 15146 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2763f4b36424b518849a07d07a5f916 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:39.138258 15146 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2763f4b36424b518849a07d07a5f916 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2763f4b36424b518849a07d07a5f916, State: Initialized, Role: FOLLOWER
I20260809 04:43:39.138859 15146 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2763f4b36424b518849a07d07a5f916 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2763f4b36424b518849a07d07a5f916" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42671 } }
I20260809 04:43:39.139089 15146 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2763f4b36424b518849a07d07a5f916 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:43:39.139187 15146 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2763f4b36424b518849a07d07a5f916 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:43:39.139336 15146 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2763f4b36424b518849a07d07a5f916 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:39.141654 15146 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2763f4b36424b518849a07d07a5f916 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2763f4b36424b518849a07d07a5f916" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42671 } }
I20260809 04:43:39.142179 15146 leader_election.cc:302] T 00000000000000000000000000000000 P e2763f4b36424b518849a07d07a5f916 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2763f4b36424b518849a07d07a5f916; no voters:
I20260809 04:43:39.142812 15146 leader_election.cc:288] T 00000000000000000000000000000000 P e2763f4b36424b518849a07d07a5f916 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:43:39.142987 15151 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2763f4b36424b518849a07d07a5f916 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:43:39.144423 15151 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2763f4b36424b518849a07d07a5f916 [term 1 LEADER]: Becoming Leader. State: Replica: e2763f4b36424b518849a07d07a5f916, State: Running, Role: LEADER
I20260809 04:43:39.145010 15151 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2763f4b36424b518849a07d07a5f916 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2763f4b36424b518849a07d07a5f916" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42671 } }
I20260809 04:43:39.145359 15146 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2763f4b36424b518849a07d07a5f916 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:43:39.148484 15153 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2763f4b36424b518849a07d07a5f916 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2763f4b36424b518849a07d07a5f916. Latest consensus state: current_term: 1 leader_uuid: "e2763f4b36424b518849a07d07a5f916" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2763f4b36424b518849a07d07a5f916" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42671 } } }
I20260809 04:43:39.149010 15153 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2763f4b36424b518849a07d07a5f916 [sys.catalog]: This master's current role is: LEADER
I20260809 04:43:39.149224 15152 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2763f4b36424b518849a07d07a5f916 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2763f4b36424b518849a07d07a5f916" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2763f4b36424b518849a07d07a5f916" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42671 } } }
I20260809 04:43:39.149561 15152 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2763f4b36424b518849a07d07a5f916 [sys.catalog]: This master's current role is: LEADER
I20260809 04:43:39.152539 15160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:43:39.158536 15160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:43:39.166985 15160 catalog_manager.cc:1347] Generated new cluster ID: 0df4384848854aa1a0bb342bb77ded72
I20260809 04:43:39.167096 15160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:43:39.176936 15160 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:43:39.178634 15160 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:43:39.189996 15160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2763f4b36424b518849a07d07a5f916: Generated new TSK 0
I20260809 04:43:39.190733 15160 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
I20260809 04:43:39.261086 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:0
--local_ip_for_outbound_sockets=127.12.234.1
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:42671
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:42285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:39.625994 15172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:39.626211 15172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:39.626295 15172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:39.632637 15172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:39.632723 15172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:39.632900 15172 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 04:43:39.632973 15172 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:43:39.633131 15172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:43:39.641141 15172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:42285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.234.62:42671
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:39.642160 15172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:39.643308 15172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:39.655246 15177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:39.655293 15180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:39.655328 15178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:39.656095 15172 server_base.cc:1034] running on GCE node
I20260809 04:43:39.656561 15172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:39.657352 15172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:39.658532 15172 hybrid_clock.cc:648] HybridClock initialized: now 1786250619658480 us; error 49 us; skew 500 ppm
I20260809 04:43:39.658784 15172 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:39 dist-test-slave-1jjb krb5kdc[15054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250619, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:39.666867 15172 init.cc:377] Logged in from keytab as kudu/127.12.234.1@KRBTEST.COM (short username kudu)
I20260809 04:43:39.668246 15172 webserver.cc:460] Webserver started at http://127.12.234.1:37361/ using document root <none> and password file <none>
I20260809 04:43:39.668848 15172 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:39.668967 15172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:39.669270 15172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:39.671725 15172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-0/data/instance:
uuid: "646783dc4be145aab61bcb438398fa88"
format_stamp: "Formatted at 2026-08-09 04:43:39 on dist-test-slave-1jjb"
server_key: "d52f03c836ae6383db64f3463b3ce48b"
server_key_iv: "025ca3246fd958cf1703e83026a24146"
server_key_version: "encryptionkey@0"
I20260809 04:43:39.672439 15172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance:
uuid: "646783dc4be145aab61bcb438398fa88"
format_stamp: "Formatted at 2026-08-09 04:43:39 on dist-test-slave-1jjb"
server_key: "d52f03c836ae6383db64f3463b3ce48b"
server_key_iv: "025ca3246fd958cf1703e83026a24146"
server_key_version: "encryptionkey@0"
I20260809 04:43:39.677086 15172 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:43:39.679888 15188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:39.681046 15172 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:43:39.681200 15172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "646783dc4be145aab61bcb438398fa88"
format_stamp: "Formatted at 2026-08-09 04:43:39 on dist-test-slave-1jjb"
server_key: "d52f03c836ae6383db64f3463b3ce48b"
server_key_iv: "025ca3246fd958cf1703e83026a24146"
server_key_version: "encryptionkey@0"
I20260809 04:43:39.681391 15172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:39.692866 15172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:39.696231 15172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:39.696465 15172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:39.698740 15172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:43:39.698859 15172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:39.698966 15172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:43:39.699041 15172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:43:39 dist-test-slave-1jjb krb5kdc[15054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250619, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:39.719735 15196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:39.699098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:54463 (local address 127.12.234.62:42671)
0809 04:43:39.699526 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:39.699533 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:39.700496 (+ 963us) server_negotiation.cc:407] Connection header received
0809 04:43:39.702267 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:39.702272 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:39.702362 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:39.702468 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:39.704295 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:39.705202 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:39.707423 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:39.707901 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:39.708015 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:39.712765 (+ 4750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:39.712770 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:39.712773 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:39.712800 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:39.716014 (+ 3214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:39.716920 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:39.716937 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:39.716942 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:39.717081 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:39.717666 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:39.717674 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:39.717679 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:39.718043 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:39.718194 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:39.719222 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0809 04:43:39.719418 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":224,"threads_started":1}
I20260809 04:43:39.721014 15198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:39.699625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:42671 (local address 127.12.234.1:54463)
0809 04:43:39.700353 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:39.700384 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:39.702027 (+ 1643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:39.702664 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:39.702671 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:39.703129 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:39.704028 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:39.704057 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:39.705362 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:39.705367 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:39.707289 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:39.707299 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:39.708123 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:39.708130 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:39.708414 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:39.709072 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:39.709123 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:39.712521 (+ 3398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:39.716148 (+ 3627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:39.716156 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:39.716170 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:39.716765 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:39.717205 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:39.717214 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:39.717216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:39.717480 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:39.718281 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:39.718292 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:39.718746 (+ 454us) client_negotiation.cc:769] Sending connection context
0809 04:43:39.719109 (+ 363us) client_negotiation.cc:240] Negotiation successful
0809 04:43:39.719526 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":301,"threads_started":1}
I20260809 04:43:39.736477 15172 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:43177
I20260809 04:43:39.736545 15311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:43177 every 8 connection(s)
I20260809 04:43:39.737744 15172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:43:39.739835 15312 heartbeater.cc:347] Connected to a master server at 127.12.234.62:42671
I20260809 04:43:39.740099 15312 heartbeater.cc:464] Registering TS with master...
I20260809 04:43:39.740820 15312 heartbeater.cc:511] Master 127.12.234.62:42671 requested a full tablet report, sending...
I20260809 04:43:39.743171 15101 ts_manager.cc:194] Registered new tserver with Master: 646783dc4be145aab61bcb438398fa88 (127.12.234.1:43177)
I20260809 04:43:39.745415 15101 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.1@KRBTEST.COM'} at 127.12.234.1:54463
I20260809 04:43:39.747689 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 15172
I20260809 04:43:39.747890 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance
I20260809 04:43:39.748014 15196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:39.735028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:33881 (local address 127.12.234.62:42671)
0809 04:43:39.735200 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:39.735207 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:39.735398 (+ 191us) server_negotiation.cc:407] Connection header received
0809 04:43:39.735616 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:39.735621 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:39.735681 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:39.735771 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:39.736919 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:39.737588 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:39.739279 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:39.739900 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:39.740031 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:39.741845 (+ 1814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:39.741851 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:39.741854 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:39.741881 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:39.745179 (+ 3298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:39.745832 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:39.745838 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:39.745841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:39.745987 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:39.746450 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:39.746456 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:39.746459 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:39.746806 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:39.746979 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:39.747382 (+ 403us) server_negotiation.cc:299] Negotiation successful
0809 04:43:39.747592 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:43:39.748055 15307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:39.734579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:42671 (local address 127.12.234.1:33881)
0809 04:43:39.735303 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:39.735322 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:39.735492 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:39.735797 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:39.735802 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:39.736365 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:39.736774 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:39.736794 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:39.737737 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:39.737743 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:39.739132 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:39.739147 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:39.740233 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:39.740247 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:39.740378 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:39.740939 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:39.740960 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:39.741637 (+ 677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:39.745289 (+ 3652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:39.745292 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:39.745295 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:39.745726 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:39.746150 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:39.746153 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:39.746155 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:39.746332 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:39.746932 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:39.746938 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:39.747124 (+ 186us) client_negotiation.cc:769] Sending connection context
0809 04:43:39.747282 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 04:43:39.747586 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":525,"threads_started":1}
I20260809 04:43:39.748741 13224 external_mini_cluster.cc:1428] Setting key ff0529e21c8449a9f14ed96c1116cea1
WARNING: no policy specified for kudu/127.12.234.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
I20260809 04:43:39.791141 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:0
--local_ip_for_outbound_sockets=127.12.234.2
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:42671
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:42285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:40.117326 15317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:40.117623 15317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:40.117729 15317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:40.125401 15317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:40.125525 15317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:40.125612 15317 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 04:43:40.125666 15317 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:43:40.125730 15317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:43:40.136201 15317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:42285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.234.62:42671
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:40.137650 15317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:40.139019 15317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:40.155571 15325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:40.155722 15322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:40.155586 15323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:40.156580 15317 server_base.cc:1034] running on GCE node
I20260809 04:43:40.157248 15317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:40.158262 15317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:40.159487 15317 hybrid_clock.cc:648] HybridClock initialized: now 1786250620159431 us; error 40 us; skew 500 ppm
I20260809 04:43:40.159855 15317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:40 dist-test-slave-1jjb krb5kdc[15054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250620, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:40.163189 15317 init.cc:377] Logged in from keytab as kudu/127.12.234.2@KRBTEST.COM (short username kudu)
I20260809 04:43:40.164968 15317 webserver.cc:460] Webserver started at http://127.12.234.2:39487/ using document root <none> and password file <none>
I20260809 04:43:40.165681 15317 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:40.165800 15317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:40.166088 15317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:40.169248 15317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-1/data/instance:
uuid: "32ac27385c334ce58e132403a2ef9711"
format_stamp: "Formatted at 2026-08-09 04:43:40 on dist-test-slave-1jjb"
server_key: "7720f0321fcaefac6fcd27ba4401593d"
server_key_iv: "6ba1dcfa5c234e7b00517cff5ece9ad6"
server_key_version: "encryptionkey@0"
I20260809 04:43:40.170022 15317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance:
uuid: "32ac27385c334ce58e132403a2ef9711"
format_stamp: "Formatted at 2026-08-09 04:43:40 on dist-test-slave-1jjb"
server_key: "7720f0321fcaefac6fcd27ba4401593d"
server_key_iv: "6ba1dcfa5c234e7b00517cff5ece9ad6"
server_key_version: "encryptionkey@0"
I20260809 04:43:40.175727 15317 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 04:43:40.179482 15333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:40.180927 15317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 04:43:40.181100 15317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "32ac27385c334ce58e132403a2ef9711"
format_stamp: "Formatted at 2026-08-09 04:43:40 on dist-test-slave-1jjb"
server_key: "7720f0321fcaefac6fcd27ba4401593d"
server_key_iv: "6ba1dcfa5c234e7b00517cff5ece9ad6"
server_key_version: "encryptionkey@0"
I20260809 04:43:40.181294 15317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:40.200666 15317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:40.205202 15317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:40.205542 15317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:40.208647 15317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:43:40.208802 15317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:40.208935 15317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:43:40.209024 15317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:43:40 dist-test-slave-1jjb krb5kdc[15054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250620, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:40.235576 15196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:40.209083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:46295 (local address 127.12.234.62:42671)
0809 04:43:40.209278 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:40.209287 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:40.210888 (+ 1601us) server_negotiation.cc:407] Connection header received
0809 04:43:40.213165 (+ 2277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:40.213171 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:40.213275 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:40.213389 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:40.215890 (+ 2501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:40.217276 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:40.220457 (+ 3181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:40.221313 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:40.221480 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:40.226862 (+ 5382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:40.226871 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:40.226878 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:40.226937 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:40.231060 (+ 4123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:40.232431 (+ 1371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:40.232439 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:40.232444 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:40.232606 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:40.233282 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:40.233287 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:40.233292 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:40.233674 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:40.233883 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:40.235123 (+ 1240us) server_negotiation.cc:299] Negotiation successful
0809 04:43:40.235311 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 04:43:40.238041 15342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:40.209833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:42671 (local address 127.12.234.2:46295)
0809 04:43:40.210724 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:40.210763 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:40.212815 (+ 2052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:40.213721 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:40.213738 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:40.214668 (+ 930us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:40.215638 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:40.215682 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:40.217487 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:40.217495 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:40.220005 (+ 2510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:40.220019 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:40.221604 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:40.221615 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:40.221961 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:40.222751 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:40.222791 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:40.226466 (+ 3675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:40.231267 (+ 4801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:40.231280 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:40.231306 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:40.232226 (+ 920us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:40.232736 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:40.232741 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:40.232745 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:40.233125 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:40.233875 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:40.233889 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:40.234571 (+ 682us) client_negotiation.cc:769] Sending connection context
0809 04:43:40.235039 (+ 468us) client_negotiation.cc:240] Negotiation successful
0809 04:43:40.235863 (+ 824us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":357,"threads_started":1}
I20260809 04:43:40.262648 15416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:40.246391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:42671 (local address 127.12.234.2:39481)
0809 04:43:40.247075 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:40.247091 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:40.247274 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:40.247663 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:40.247668 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:40.248272 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:40.248782 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:40.248800 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:40.250249 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:40.250254 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:40.251548 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:40.251561 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:40.252393 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:40.252400 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:40.252583 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:40.253397 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:40.253415 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:40.254305 (+ 890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:40.259150 (+ 4845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:40.259156 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:40.259161 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:40.259812 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:40.260287 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:40.260290 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:40.260293 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:40.260439 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:40.261258 (+ 819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:40.261263 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:40.261520 (+ 257us) client_negotiation.cc:769] Sending connection context
0809 04:43:40.261727 (+ 207us) client_negotiation.cc:240] Negotiation successful
0809 04:43:40.262105 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":394,"threads_started":1}
I20260809 04:43:40.263401 15196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:40.246286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:39481 (local address 127.12.234.62:42671)
0809 04:43:40.246546 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:40.246551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:40.247187 (+ 636us) server_negotiation.cc:407] Connection header received
0809 04:43:40.247434 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:40.247439 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:40.247504 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:40.247631 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:40.248957 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:40.250101 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:40.251690 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:40.252197 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:40.252343 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:40.254551 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:40.254561 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:40.254574 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:40.254641 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:40.258954 (+ 4313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:40.259955 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:40.259960 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:40.259963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:40.260173 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:40.260558 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:40.260564 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:40.260567 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:40.261083 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:40.261243 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:40.263013 (+ 1770us) server_negotiation.cc:299] Negotiation successful
0809 04:43:40.263193 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 04:43:40.270138 15317 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:46829
I20260809 04:43:40.270176 15455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:46829 every 8 connection(s)
I20260809 04:43:40.271458 15317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:43:40.274200 15456 heartbeater.cc:347] Connected to a master server at 127.12.234.62:42671
I20260809 04:43:40.274566 15456 heartbeater.cc:464] Registering TS with master...
I20260809 04:43:40.275540 15456 heartbeater.cc:511] Master 127.12.234.62:42671 requested a full tablet report, sending...
I20260809 04:43:40.277426 15100 ts_manager.cc:194] Registered new tserver with Master: 32ac27385c334ce58e132403a2ef9711 (127.12.234.2:46829)
I20260809 04:43:40.278472 15100 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.2@KRBTEST.COM'} at 127.12.234.2:46295
I20260809 04:43:40.279390 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 15317
I20260809 04:43:40.279559 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance
I20260809 04:43:40.280167 13224 external_mini_cluster.cc:1428] Setting key 5d0ada1835e0c58645e70d906e2b7317
WARNING: no policy specified for kudu/127.12.234.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
I20260809 04:43:40.334021 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:0
--local_ip_for_outbound_sockets=127.12.234.3
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:42671
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:42285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:40.697996 15461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:40.698346 15461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:40.698526 15461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:40.706144 15461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:40.706326 15461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:40.706521 15461 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 04:43:40.706620 15461 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:43:40.706755 15461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:43:40.717236 15461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:42285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.234.62:42671
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:40.718778 15461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:40.720288 15461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:40.734840 15466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:40.734776 15469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:40.734799 15467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:40.736025 15461 server_base.cc:1034] running on GCE node
I20260809 04:43:40.736726 15461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:40.737718 15461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:40.738955 15461 hybrid_clock.cc:648] HybridClock initialized: now 1786250620738901 us; error 52 us; skew 500 ppm
I20260809 04:43:40.739290 15461 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:40 dist-test-slave-1jjb krb5kdc[15054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250620, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:40.742666 15461 init.cc:377] Logged in from keytab as kudu/127.12.234.3@KRBTEST.COM (short username kudu)
I20260809 04:43:40.744273 15461 webserver.cc:460] Webserver started at http://127.12.234.3:33959/ using document root <none> and password file <none>
I20260809 04:43:40.744992 15461 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:40.745164 15461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:40.745486 15461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:40.748838 15312 heartbeater.cc:503] Master 127.12.234.62:42671 was elected leader, sending a full tablet report...
I20260809 04:43:40.748793 15461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-2/data/instance:
uuid: "477443567ac448b4a2595113d452ef22"
format_stamp: "Formatted at 2026-08-09 04:43:40 on dist-test-slave-1jjb"
server_key: "26ca8a276b9fb85d1ee5a29eb19fae73"
server_key_iv: "dd61bc8231302d685fce63cb75b0728b"
server_key_version: "encryptionkey@0"
I20260809 04:43:40.749624 15461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance:
uuid: "477443567ac448b4a2595113d452ef22"
format_stamp: "Formatted at 2026-08-09 04:43:40 on dist-test-slave-1jjb"
server_key: "26ca8a276b9fb85d1ee5a29eb19fae73"
server_key_iv: "dd61bc8231302d685fce63cb75b0728b"
server_key_version: "encryptionkey@0"
I20260809 04:43:40.754997 15461 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 04:43:40.759654 15476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:40.761507 15461 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 04:43:40.761677 15461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "477443567ac448b4a2595113d452ef22"
format_stamp: "Formatted at 2026-08-09 04:43:40 on dist-test-slave-1jjb"
server_key: "26ca8a276b9fb85d1ee5a29eb19fae73"
server_key_iv: "dd61bc8231302d685fce63cb75b0728b"
server_key_version: "encryptionkey@0"
I20260809 04:43:40.761880 15461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:40.808959 15461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:40.814248 15461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:40.814584 15461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:40.818756 15461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:43:40.818902 15461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:40.819067 15461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:43:40.819193 15461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:43:40 dist-test-slave-1jjb krb5kdc[15054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250620, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:40.857419 15487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:40.821311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:42671 (local address 127.12.234.3:33791)
0809 04:43:40.822661 (+ 1350us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:40.822771 (+ 110us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:40.826675 (+ 3904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:40.827873 (+ 1198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:40.827882 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:40.828656 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:40.829635 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:40.829661 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:40.831339 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:40.831349 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:40.834489 (+ 3140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:40.834546 (+ 57us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:40.836247 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:40.836262 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:40.837036 (+ 774us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:40.838200 (+ 1164us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:40.838232 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:40.842103 (+ 3871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:40.847818 (+ 5715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:40.847831 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:40.847862 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:40.849249 (+ 1387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:40.850099 (+ 850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:40.850108 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:40.850117 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:40.850505 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:40.851747 (+ 1242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:40.851766 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:40.852846 (+ 1080us) client_negotiation.cc:769] Sending connection context
0809 04:43:40.853418 (+ 572us) client_negotiation.cc:240] Negotiation successful
0809 04:43:40.854519 (+ 1101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":861,"thread_start_us":514,"threads_started":1}
I20260809 04:43:40.858460 15486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:40.820061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:33791 (local address 127.12.234.62:42671)
0809 04:43:40.821138 (+ 1077us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:40.821146 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:40.823002 (+ 1856us) server_negotiation.cc:407] Connection header received
0809 04:43:40.827219 (+ 4217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:40.827228 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:40.827376 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:40.827542 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:40.829881 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:40.831071 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:40.834808 (+ 3737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:40.835488 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:40.835880 (+ 392us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:40.842522 (+ 6642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:40.842546 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:40.842554 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:40.842627 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:40.847585 (+ 4958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:40.849634 (+ 2049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:40.849645 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:40.849651 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:40.849898 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:40.850732 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:40.850743 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:40.850749 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:40.851263 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:40.851593 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:40.857294 (+ 5701us) server_negotiation.cc:299] Negotiation successful
0809 04:43:40.857810 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":860,"thread_start_us":703,"threads_started":1}
I20260809 04:43:40.876374 15461 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:40143
I20260809 04:43:40.876430 15602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:40143 every 8 connection(s)
I20260809 04:43:40.878413 15461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:43:40.878906 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 15461
I20260809 04:43:40.879135 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance
I20260809 04:43:40.880659 13224 external_mini_cluster.cc:1428] Setting key 0ce0a00d41b5927734cf88b49bb58459
I20260809 04:43:40.882400 15603 heartbeater.cc:347] Connected to a master server at 127.12.234.62:42671
I20260809 04:43:40.882673 15603 heartbeater.cc:464] Registering TS with master...
I20260809 04:43:40.883744 15603 heartbeater.cc:511] Master 127.12.234.62:42671 requested a full tablet report, sending...
I20260809 04:43:40.885880 15101 ts_manager.cc:194] Registered new tserver with Master: 477443567ac448b4a2595113d452ef22 (127.12.234.3:40143)
I20260809 04:43:40.887115 15486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:40.867853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:57179 (local address 127.12.234.62:42671)
0809 04:43:40.868121 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:40.868128 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:40.869051 (+ 923us) server_negotiation.cc:407] Connection header received
0809 04:43:40.869383 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:40.869393 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:40.869524 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:40.869704 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:40.871561 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:40.873075 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:40.875160 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:40.876137 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:40.876335 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:40.878778 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:40.878784 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:40.878787 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:40.878825 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:40.883435 (+ 4610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:40.884646 (+ 1211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:40.884655 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:40.884659 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:40.884887 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:40.885471 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:40.885476 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:40.885480 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:40.885843 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:40.885969 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:40.886632 (+ 663us) server_negotiation.cc:299] Negotiation successful
0809 04:43:40.886844 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 04:43:40.887205 15101 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.3@KRBTEST.COM'} at 127.12.234.3:33791
I20260809 04:43:40.887502 15585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:40.868112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:42671 (local address 127.12.234.3:57179)
0809 04:43:40.868866 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:40.868893 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:40.869139 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:40.869749 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:40.869756 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:40.870639 (+ 883us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:40.871288 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:40.871310 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:40.873296 (+ 1986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:40.873305 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:40.874932 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:40.874952 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:40.876463 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:40.876474 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:40.876719 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:40.877566 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:40.877585 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:40.878514 (+ 929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:40.883617 (+ 5103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:40.883624 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:40.883629 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:40.884488 (+ 859us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:40.885082 (+ 594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:40.885090 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:40.885096 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:40.885293 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:40.886018 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:40.886024 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:40.886313 (+ 289us) client_negotiation.cc:769] Sending connection context
0809 04:43:40.886536 (+ 223us) client_negotiation.cc:240] Negotiation successful
0809 04:43:40.886929 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":411,"threads_started":1}
I20260809 04:43:40.898192 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 04:43:40 dist-test-slave-1jjb krb5kdc[15054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 04:43:40 dist-test-slave-1jjb krb5kdc[15054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:40.970122 15486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:40.950315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38632 (local address 127.12.234.62:42671)
0809 04:43:40.950560 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:40.950566 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:40.950891 (+ 325us) server_negotiation.cc:407] Connection header received
0809 04:43:40.951108 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:40.951113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:40.951206 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:40.951353 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:40.954404 (+ 3051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:40.955358 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:40.957252 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:40.957873 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:40.958160 (+ 287us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:40.963442 (+ 5282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:40.963450 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:40.963454 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:40.963519 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:40.967110 (+ 3591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:40.967932 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:40.967937 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:40.967940 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:40.968184 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:40.968548 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:40.968551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:40.968553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:40.968852 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:40.969096 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:40.969495 (+ 399us) server_negotiation.cc:299] Negotiation successful
0809 04:43:40.969698 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 04:43:40.975870 15101 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38632:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:43:40.983589 15101 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:43:41.018111 15622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:40.996984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:46829 (local address 127.0.0.1:50914)
0809 04:43:41.009650 (+ 12666us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:41.009666 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:41.009760 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:41.010565 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:41.010571 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:41.010592 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:41.011205 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:41.011217 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.013056 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.013060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:41.015208 (+ 2148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:41.015221 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.016979 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.016985 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:41.017212 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:41.017273 (+ 61us) client_negotiation.cc:769] Sending connection context
0809 04:43:41.017414 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 04:43:41.017606 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11870,"mutex_wait_us":110,"thread_start_us":10737,"threads_started":1}
I20260809 04:43:41.018211 15618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:40.996839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:40143 (local address 127.0.0.1:45510)
0809 04:43:41.008110 (+ 11271us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:41.008145 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:41.008349 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:41.010234 (+ 1885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:41.010242 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:41.010304 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:41.011054 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:41.011072 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.012806 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.012811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:41.015376 (+ 2565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:41.015390 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.017319 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.017325 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:41.017530 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:41.017562 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 04:43:41.017654 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 04:43:41.017773 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11024,"mutex_wait_us":29,"spinlock_wait_cycles":26368,"thread_start_us":12242,"threads_started":1}
I20260809 04:43:41.018656 15617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:40.999894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:43177 (local address 127.0.0.1:48502)
0809 04:43:41.009353 (+ 9459us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:41.009379 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:41.009514 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:41.010236 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:41.010241 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:41.010290 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:41.011029 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:41.011052 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.013051 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.013056 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:41.015646 (+ 2590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:41.015658 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.017648 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.017655 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:41.017822 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:41.017859 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 04:43:41.017972 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 04:43:41.018099 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9324,"mutex_wait_us":80,"spinlock_wait_cycles":1792,"thread_start_us":8778,"threads_started":1}
I20260809 04:43:41.018880 15620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:40.997265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50914 (local address 127.12.234.2:46829)
0809 04:43:41.008947 (+ 11682us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:41.008959 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:41.009868 (+ 909us) server_negotiation.cc:407] Connection header received
0809 04:43:41.009989 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:41.009996 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:41.010300 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:41.010555 (+ 255us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:41.011391 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.012880 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.015394 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.016771 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.016934 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:41.017158 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:41.018411 (+ 1253us) server_negotiation.cc:299] Negotiation successful
0809 04:43:41.018593 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11505,"thread_start_us":11672,"threads_started":1}
I20260809 04:43:41.019146 15619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:40.997203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45510 (local address 127.12.234.3:40143)
0809 04:43:41.009586 (+ 12383us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:41.009596 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:41.009629 (+ 33us) server_negotiation.cc:407] Connection header received
0809 04:43:41.009735 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:41.009742 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:41.009995 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:41.010197 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:41.011317 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.012645 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.015558 (+ 2913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.017044 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.017177 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:41.017403 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:41.018658 (+ 1255us) server_negotiation.cc:299] Negotiation successful
0809 04:43:41.018805 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12222,"thread_start_us":12020,"threads_started":1}
I20260809 04:43:41.019366 15623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:41.000670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48502 (local address 127.12.234.1:43177)
0809 04:43:41.005007 (+ 4337us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:41.005020 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:41.009511 (+ 4491us) server_negotiation.cc:407] Connection header received
0809 04:43:41.009702 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:41.009712 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:41.009992 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:41.010815 (+ 823us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:41.011209 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.012886 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.015855 (+ 2969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.017365 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.017583 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:41.017776 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:41.018041 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 04:43:41.019067 (+ 1026us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4126,"thread_start_us":3986,"threads_started":1}
I20260809 04:43:41.022662 15529 tablet_service.cc:1467] Processing CreateTablet for tablet 6642ce2983574784a92d4554c329493b (DEFAULT_TABLE table=test-table [id=72de9e6b2a4f4c8ebb40365c998c2523]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:43:41.022675 15240 tablet_service.cc:1467] Processing CreateTablet for tablet 6642ce2983574784a92d4554c329493b (DEFAULT_TABLE table=test-table [id=72de9e6b2a4f4c8ebb40365c998c2523]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:43:41.022673 15384 tablet_service.cc:1467] Processing CreateTablet for tablet 6642ce2983574784a92d4554c329493b (DEFAULT_TABLE table=test-table [id=72de9e6b2a4f4c8ebb40365c998c2523]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:43:41.024099 15240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6642ce2983574784a92d4554c329493b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:41.024115 15384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6642ce2983574784a92d4554c329493b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:41.024137 15529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6642ce2983574784a92d4554c329493b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:41.035666 15624 tablet_bootstrap.cc:492] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22: Bootstrap starting.
I20260809 04:43:41.039191 15625 tablet_bootstrap.cc:492] T 6642ce2983574784a92d4554c329493b P 32ac27385c334ce58e132403a2ef9711: Bootstrap starting.
I20260809 04:43:41.039508 15624 tablet_bootstrap.cc:654] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:41.040889 15626 tablet_bootstrap.cc:492] T 6642ce2983574784a92d4554c329493b P 646783dc4be145aab61bcb438398fa88: Bootstrap starting.
I20260809 04:43:41.041054 15624 log.cc:824] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22: Log is configured to *not* fsync() on all Append() calls
I20260809 04:43:41.042508 15625 tablet_bootstrap.cc:654] T 6642ce2983574784a92d4554c329493b P 32ac27385c334ce58e132403a2ef9711: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:41.043772 15625 log.cc:824] T 6642ce2983574784a92d4554c329493b P 32ac27385c334ce58e132403a2ef9711: Log is configured to *not* fsync() on all Append() calls
I20260809 04:43:41.043957 15626 tablet_bootstrap.cc:654] T 6642ce2983574784a92d4554c329493b P 646783dc4be145aab61bcb438398fa88: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:41.044517 15624 tablet_bootstrap.cc:492] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22: No bootstrap required, opened a new log
I20260809 04:43:41.044802 15624 ts_tablet_manager.cc:1397] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260809 04:43:41.045240 15626 log.cc:824] T 6642ce2983574784a92d4554c329493b P 646783dc4be145aab61bcb438398fa88: Log is configured to *not* fsync() on all Append() calls
I20260809 04:43:41.047163 15625 tablet_bootstrap.cc:492] T 6642ce2983574784a92d4554c329493b P 32ac27385c334ce58e132403a2ef9711: No bootstrap required, opened a new log
I20260809 04:43:41.047446 15625 ts_tablet_manager.cc:1397] T 6642ce2983574784a92d4554c329493b P 32ac27385c334ce58e132403a2ef9711: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 04:43:41.048220 15626 tablet_bootstrap.cc:492] T 6642ce2983574784a92d4554c329493b P 646783dc4be145aab61bcb438398fa88: No bootstrap required, opened a new log
I20260809 04:43:41.048467 15626 ts_tablet_manager.cc:1397] T 6642ce2983574784a92d4554c329493b P 646783dc4be145aab61bcb438398fa88: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 04:43:41.055706 15626 raft_consensus.cc:348] T 6642ce2983574784a92d4554c329493b P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } }
I20260809 04:43:41.056051 15626 raft_consensus.cc:374] T 6642ce2983574784a92d4554c329493b P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:41.056192 15626 raft_consensus.cc:729] T 6642ce2983574784a92d4554c329493b P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 646783dc4be145aab61bcb438398fa88, State: Initialized, Role: FOLLOWER
I20260809 04:43:41.056286 15624 raft_consensus.cc:348] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } }
I20260809 04:43:41.056743 15624 raft_consensus.cc:374] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:41.056859 15626 consensus_queue.cc:260] T 6642ce2983574784a92d4554c329493b P 646783dc4be145aab61bcb438398fa88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } }
I20260809 04:43:41.056841 15624 raft_consensus.cc:729] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 477443567ac448b4a2595113d452ef22, State: Initialized, Role: FOLLOWER
I20260809 04:43:41.057968 15624 consensus_queue.cc:260] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } }
I20260809 04:43:41.058955 15625 raft_consensus.cc:348] T 6642ce2983574784a92d4554c329493b P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } }
I20260809 04:43:41.059369 15626 ts_tablet_manager.cc:1428] T 6642ce2983574784a92d4554c329493b P 646783dc4be145aab61bcb438398fa88: Time spent starting tablet: real 0.011s user 0.006s sys 0.004s
I20260809 04:43:41.059501 15625 raft_consensus.cc:374] T 6642ce2983574784a92d4554c329493b P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:41.059628 15625 raft_consensus.cc:729] T 6642ce2983574784a92d4554c329493b P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32ac27385c334ce58e132403a2ef9711, State: Initialized, Role: FOLLOWER
I20260809 04:43:41.060316 15625 consensus_queue.cc:260] T 6642ce2983574784a92d4554c329493b P 32ac27385c334ce58e132403a2ef9711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } }
I20260809 04:43:41.061266 15624 ts_tablet_manager.cc:1428] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22: Time spent starting tablet: real 0.016s user 0.016s sys 0.001s
I20260809 04:43:41.061751 15603 heartbeater.cc:503] Master 127.12.234.62:42671 was elected leader, sending a full tablet report...
I20260809 04:43:41.061892 15456 heartbeater.cc:503] Master 127.12.234.62:42671 was elected leader, sending a full tablet report...
I20260809 04:43:41.062096 15625 ts_tablet_manager.cc:1428] T 6642ce2983574784a92d4554c329493b P 32ac27385c334ce58e132403a2ef9711: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260809 04:43:41.129604 15631 raft_consensus.cc:482] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:43:41.130088 15631 raft_consensus.cc:504] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } }
W20260809 04:43:41.130925 15604 tablet.cc:2369] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:43:41.132441 15631 leader_election.cc:288] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32ac27385c334ce58e132403a2ef9711 (127.12.234.2:46829), 646783dc4be145aab61bcb438398fa88 (127.12.234.1:43177)
I20260809 04:43:41.138885 15620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:41.132753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:59239 (local address 127.12.234.2:46829)
0809 04:43:41.133051 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:41.133058 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:41.133111 (+ 53us) server_negotiation.cc:407] Connection header received
0809 04:43:41.133394 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:41.133398 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:41.133464 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:41.133569 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:41.134024 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.135055 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.136976 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.138074 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.138200 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:41.138252 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:41.138599 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 04:43:41.138693 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260809 04:43:41.139223 15487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:41.132675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:46829 (local address 127.12.234.3:59239)
0809 04:43:41.133036 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:41.133057 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:41.133198 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:41.133594 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:41.133599 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:41.133627 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:41.133866 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:41.133878 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.135210 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.135216 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:41.136811 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:41.136823 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.138227 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.138234 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:41.138428 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:41.138465 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 04:43:41.138569 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 04:43:41.138710 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":20}
I20260809 04:43:41.139822 15623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:41.133062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:49923 (local address 127.12.234.1:43177)
0809 04:43:41.133313 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:41.133321 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:41.133798 (+ 477us) server_negotiation.cc:407] Connection header received
0809 04:43:41.133947 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:41.133954 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:41.134042 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:41.134111 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:41.134716 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.135628 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.137776 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.138844 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.139003 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:41.139083 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:41.139516 (+ 433us) server_negotiation.cc:299] Negotiation successful
0809 04:43:41.139631 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260809 04:43:41.139999 15633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:41.132909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:43177 (local address 127.12.234.3:49923)
0809 04:43:41.133694 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:41.133708 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:41.133819 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:41.134171 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:41.134176 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:41.134202 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:41.134570 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:41.134582 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.135795 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.135799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:41.137604 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:41.137620 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.139069 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.139077 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:41.139251 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:41.139312 (+ 61us) client_negotiation.cc:769] Sending connection context
0809 04:43:41.139457 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 04:43:41.139595 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"mutex_wait_us":44,"thread_start_us":443,"threads_started":1}
I20260809 04:43:41.140450 15406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6642ce2983574784a92d4554c329493b" candidate_uuid: "477443567ac448b4a2595113d452ef22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32ac27385c334ce58e132403a2ef9711" is_pre_election: true
I20260809 04:43:41.141134 15406 raft_consensus.cc:2454] T 6642ce2983574784a92d4554c329493b P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 477443567ac448b4a2595113d452ef22 in term 0.
I20260809 04:43:41.141090 15262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6642ce2983574784a92d4554c329493b" candidate_uuid: "477443567ac448b4a2595113d452ef22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "646783dc4be145aab61bcb438398fa88" is_pre_election: true
I20260809 04:43:41.141635 15262 raft_consensus.cc:2454] T 6642ce2983574784a92d4554c329493b P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 477443567ac448b4a2595113d452ef22 in term 0.
I20260809 04:43:41.141903 15477 leader_election.cc:302] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32ac27385c334ce58e132403a2ef9711, 477443567ac448b4a2595113d452ef22; no voters:
I20260809 04:43:41.142369 15631 raft_consensus.cc:2781] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:43:41.142539 15631 raft_consensus.cc:482] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:43:41.142652 15631 raft_consensus.cc:3037] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:41.144948 15631 raft_consensus.cc:504] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } }
I20260809 04:43:41.145558 15631 leader_election.cc:288] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22 [CANDIDATE]: Term 1 election: Requested vote from peers 32ac27385c334ce58e132403a2ef9711 (127.12.234.2:46829), 646783dc4be145aab61bcb438398fa88 (127.12.234.1:43177)
I20260809 04:43:41.146025 15406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6642ce2983574784a92d4554c329493b" candidate_uuid: "477443567ac448b4a2595113d452ef22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32ac27385c334ce58e132403a2ef9711"
I20260809 04:43:41.146106 15262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6642ce2983574784a92d4554c329493b" candidate_uuid: "477443567ac448b4a2595113d452ef22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "646783dc4be145aab61bcb438398fa88"
I20260809 04:43:41.146251 15406 raft_consensus.cc:3037] T 6642ce2983574784a92d4554c329493b P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:41.146282 15262 raft_consensus.cc:3037] T 6642ce2983574784a92d4554c329493b P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:41.149701 15262 raft_consensus.cc:2454] T 6642ce2983574784a92d4554c329493b P 646783dc4be145aab61bcb438398fa88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 477443567ac448b4a2595113d452ef22 in term 1.
I20260809 04:43:41.149701 15406 raft_consensus.cc:2454] T 6642ce2983574784a92d4554c329493b P 32ac27385c334ce58e132403a2ef9711 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 477443567ac448b4a2595113d452ef22 in term 1.
I20260809 04:43:41.150187 15480 leader_election.cc:302] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 477443567ac448b4a2595113d452ef22, 646783dc4be145aab61bcb438398fa88; no voters:
I20260809 04:43:41.150635 15631 raft_consensus.cc:2781] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:43:41.151117 15631 raft_consensus.cc:686] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22 [term 1 LEADER]: Becoming Leader. State: Replica: 477443567ac448b4a2595113d452ef22, State: Running, Role: LEADER
I20260809 04:43:41.151629 15631 consensus_queue.cc:237] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } }
I20260809 04:43:41.156754 15100 catalog_manager.cc:5495] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22 reported cstate change: term changed from 0 to 1, leader changed from <none> to 477443567ac448b4a2595113d452ef22 (127.12.234.3). New cstate: current_term: 1 leader_uuid: "477443567ac448b4a2595113d452ef22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } health_report { overall_health: UNKNOWN } } }
Aug 09 04:43:41 dist-test-slave-1jjb krb5kdc[15054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250618, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:41.209969 15642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:41.195073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:42671 (local address 127.0.0.1:38638)
0809 04:43:41.195864 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:41.195881 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:41.196024 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:41.196486 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:41.196490 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:41.197261 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:41.197527 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:41.197548 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.198554 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.198561 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:41.199550 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:41.199561 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.200406 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.200412 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:41.200560 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:41.201322 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:41.201340 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:41.203737 (+ 2397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:41.206968 (+ 3231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:41.206985 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:41.206994 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:41.207495 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:41.207867 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:41.207872 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:41.207877 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:41.208179 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:41.208792 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:41.208801 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:41.208962 (+ 161us) client_negotiation.cc:769] Sending connection context
0809 04:43:41.209148 (+ 186us) client_negotiation.cc:240] Negotiation successful
0809 04:43:41.209460 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":457,"threads_started":1}
I20260809 04:43:41.209967 15486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:41.195062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38638 (local address 127.12.234.62:42671)
0809 04:43:41.195355 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:41.195360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:41.195996 (+ 636us) server_negotiation.cc:407] Connection header received
0809 04:43:41.196219 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:41.196227 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:41.196316 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:41.196465 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:41.197682 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.198423 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.199704 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.200199 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.200329 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:41.203962 (+ 3633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:41.203967 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:41.203970 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:41.203999 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:41.206798 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:41.207619 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:41.207627 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:41.207630 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:41.207755 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:41.208296 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:41.208303 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:41.208307 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:41.208656 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:41.208809 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:41.209239 (+ 430us) server_negotiation.cc:299] Negotiation successful
0809 04:43:41.209604 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":143,"server-negotiator.queue_time_us":86}
I20260809 04:43:41.215906 15099 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:38638:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 04:43:41.217183 15099 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:43:41.225164 15240 tablet_service.cc:1467] Processing CreateTablet for tablet afb20966d6b440b186199f0362061263 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2c750d9434fe4f9f8420692e7655a982]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:43:41.225487 15529 tablet_service.cc:1467] Processing CreateTablet for tablet afb20966d6b440b186199f0362061263 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2c750d9434fe4f9f8420692e7655a982]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:43:41.225605 15384 tablet_service.cc:1467] Processing CreateTablet for tablet afb20966d6b440b186199f0362061263 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2c750d9434fe4f9f8420692e7655a982]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:43:41.226300 15529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afb20966d6b440b186199f0362061263. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:41.226303 15240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afb20966d6b440b186199f0362061263. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:41.226440 15384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afb20966d6b440b186199f0362061263. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:41.232384 15624 tablet_bootstrap.cc:492] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22: Bootstrap starting.
I20260809 04:43:41.233887 15625 tablet_bootstrap.cc:492] T afb20966d6b440b186199f0362061263 P 32ac27385c334ce58e132403a2ef9711: Bootstrap starting.
I20260809 04:43:41.233901 15626 tablet_bootstrap.cc:492] T afb20966d6b440b186199f0362061263 P 646783dc4be145aab61bcb438398fa88: Bootstrap starting.
I20260809 04:43:41.234223 15624 tablet_bootstrap.cc:654] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:41.235597 15626 tablet_bootstrap.cc:654] T afb20966d6b440b186199f0362061263 P 646783dc4be145aab61bcb438398fa88: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:41.235621 15625 tablet_bootstrap.cc:654] T afb20966d6b440b186199f0362061263 P 32ac27385c334ce58e132403a2ef9711: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:41.238456 15624 tablet_bootstrap.cc:492] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22: No bootstrap required, opened a new log
I20260809 04:43:41.238636 15624 ts_tablet_manager.cc:1397] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 04:43:41.239602 15624 raft_consensus.cc:348] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:41.239831 15624 raft_consensus.cc:374] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:41.239939 15624 raft_consensus.cc:729] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 477443567ac448b4a2595113d452ef22, State: Initialized, Role: FOLLOWER
I20260809 04:43:41.240288 15626 tablet_bootstrap.cc:492] T afb20966d6b440b186199f0362061263 P 646783dc4be145aab61bcb438398fa88: No bootstrap required, opened a new log
I20260809 04:43:41.240281 15625 tablet_bootstrap.cc:492] T afb20966d6b440b186199f0362061263 P 32ac27385c334ce58e132403a2ef9711: No bootstrap required, opened a new log
I20260809 04:43:41.240437 15625 ts_tablet_manager.cc:1397] T afb20966d6b440b186199f0362061263 P 32ac27385c334ce58e132403a2ef9711: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260809 04:43:41.240284 15624 consensus_queue.cc:260] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:41.240442 15626 ts_tablet_manager.cc:1397] T afb20966d6b440b186199f0362061263 P 646783dc4be145aab61bcb438398fa88: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
W20260809 04:43:41.240419 15313 tablet.cc:2369] T 6642ce2983574784a92d4554c329493b P 646783dc4be145aab61bcb438398fa88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:43:41.240759 15631 tablet_replica.cc:439] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } } }
I20260809 04:43:41.241039 15631 tablet_replica.cc:442] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:43:41.241335 15626 raft_consensus.cc:348] T afb20966d6b440b186199f0362061263 P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:41.241323 15625 raft_consensus.cc:348] T afb20966d6b440b186199f0362061263 P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:41.241560 15626 raft_consensus.cc:374] T afb20966d6b440b186199f0362061263 P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:41.241580 15625 raft_consensus.cc:374] T afb20966d6b440b186199f0362061263 P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:41.241658 15626 raft_consensus.cc:729] T afb20966d6b440b186199f0362061263 P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 646783dc4be145aab61bcb438398fa88, State: Initialized, Role: FOLLOWER
I20260809 04:43:41.241669 15625 raft_consensus.cc:729] T afb20966d6b440b186199f0362061263 P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32ac27385c334ce58e132403a2ef9711, State: Initialized, Role: FOLLOWER
I20260809 04:43:41.241917 15626 consensus_queue.cc:260] T afb20966d6b440b186199f0362061263 P 646783dc4be145aab61bcb438398fa88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:41.242246 15624 ts_tablet_manager.cc:1428] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 04:43:41.242465 15626 ts_tablet_manager.cc:1428] T afb20966d6b440b186199f0362061263 P 646783dc4be145aab61bcb438398fa88: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 04:43:41.243286 15630 tablet_replica.cc:439] T afb20966d6b440b186199f0362061263 P 646783dc4be145aab61bcb438398fa88: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } } }
I20260809 04:43:41.243598 15630 tablet_replica.cc:442] T afb20966d6b440b186199f0362061263 P 646783dc4be145aab61bcb438398fa88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:43:41.244323 15625 consensus_queue.cc:260] T afb20966d6b440b186199f0362061263 P 32ac27385c334ce58e132403a2ef9711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:41.245024 15625 ts_tablet_manager.cc:1428] T afb20966d6b440b186199f0362061263 P 32ac27385c334ce58e132403a2ef9711: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 04:43:41.244808 15632 tablet_replica.cc:439] T afb20966d6b440b186199f0362061263 P 32ac27385c334ce58e132403a2ef9711: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } } }
I20260809 04:43:41.245256 15632 tablet_replica.cc:442] T afb20966d6b440b186199f0362061263 P 32ac27385c334ce58e132403a2ef9711: This TxnStatusTablet replica's current role is: FOLLOWER
W20260809 04:43:41.274334 15457 tablet.cc:2369] T 6642ce2983574784a92d4554c329493b P 32ac27385c334ce58e132403a2ef9711: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:43:41.283892 15631 raft_consensus.cc:482] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:43:41.284209 15631 raft_consensus.cc:504] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:41.284857 15631 leader_election.cc:288] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 646783dc4be145aab61bcb438398fa88 (127.12.234.1:43177), 32ac27385c334ce58e132403a2ef9711 (127.12.234.2:46829)
I20260809 04:43:41.285537 15262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afb20966d6b440b186199f0362061263" candidate_uuid: "477443567ac448b4a2595113d452ef22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "646783dc4be145aab61bcb438398fa88" is_pre_election: true
I20260809 04:43:41.285542 15406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afb20966d6b440b186199f0362061263" candidate_uuid: "477443567ac448b4a2595113d452ef22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32ac27385c334ce58e132403a2ef9711" is_pre_election: true
I20260809 04:43:41.285830 15406 raft_consensus.cc:2454] T afb20966d6b440b186199f0362061263 P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 477443567ac448b4a2595113d452ef22 in term 0.
I20260809 04:43:41.285830 15262 raft_consensus.cc:2454] T afb20966d6b440b186199f0362061263 P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 477443567ac448b4a2595113d452ef22 in term 0.
I20260809 04:43:41.286283 15477 leader_election.cc:302] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 32ac27385c334ce58e132403a2ef9711, 477443567ac448b4a2595113d452ef22, 646783dc4be145aab61bcb438398fa88; no voters:
I20260809 04:43:41.286577 15631 raft_consensus.cc:2781] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:43:41.286698 15631 raft_consensus.cc:482] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:43:41.286808 15631 raft_consensus.cc:3037] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:41.289120 15631 raft_consensus.cc:504] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:41.289773 15631 leader_election.cc:288] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22 [CANDIDATE]: Term 1 election: Requested vote from peers 646783dc4be145aab61bcb438398fa88 (127.12.234.1:43177), 32ac27385c334ce58e132403a2ef9711 (127.12.234.2:46829)
I20260809 04:43:41.290289 15406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afb20966d6b440b186199f0362061263" candidate_uuid: "477443567ac448b4a2595113d452ef22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32ac27385c334ce58e132403a2ef9711"
I20260809 04:43:41.290293 15262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afb20966d6b440b186199f0362061263" candidate_uuid: "477443567ac448b4a2595113d452ef22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "646783dc4be145aab61bcb438398fa88"
I20260809 04:43:41.290535 15406 raft_consensus.cc:3037] T afb20966d6b440b186199f0362061263 P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:41.290547 15262 raft_consensus.cc:3037] T afb20966d6b440b186199f0362061263 P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:41.292900 15262 raft_consensus.cc:2454] T afb20966d6b440b186199f0362061263 P 646783dc4be145aab61bcb438398fa88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 477443567ac448b4a2595113d452ef22 in term 1.
I20260809 04:43:41.293027 15406 raft_consensus.cc:2454] T afb20966d6b440b186199f0362061263 P 32ac27385c334ce58e132403a2ef9711 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 477443567ac448b4a2595113d452ef22 in term 1.
I20260809 04:43:41.293311 15480 leader_election.cc:302] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 477443567ac448b4a2595113d452ef22, 646783dc4be145aab61bcb438398fa88; no voters:
I20260809 04:43:41.293637 15631 raft_consensus.cc:2781] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:43:41.293816 15631 raft_consensus.cc:686] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22 [term 1 LEADER]: Becoming Leader. State: Replica: 477443567ac448b4a2595113d452ef22, State: Running, Role: LEADER
I20260809 04:43:41.294112 15631 consensus_queue.cc:237] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:41.295734 15634 tablet_replica.cc:439] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22: TxnStatusTablet state changed. Reason: New leader 477443567ac448b4a2595113d452ef22. Latest consensus state: current_term: 1 leader_uuid: "477443567ac448b4a2595113d452ef22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } } }
I20260809 04:43:41.296160 15634 tablet_replica.cc:442] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:43:41.296813 15647 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 04:43:41.297103 15647 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 04:43:41.297246 15099 catalog_manager.cc:5495] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22 reported cstate change: term changed from 0 to 1, leader changed from <none> to 477443567ac448b4a2595113d452ef22 (127.12.234.3). New cstate: current_term: 1 leader_uuid: "477443567ac448b4a2595113d452ef22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } health_report { overall_health: HEALTHY } } }
I20260809 04:43:41.324365 15642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:41.317220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:40143 (local address 127.0.0.1:39180)
0809 04:43:41.317513 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:41.317531 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:41.317651 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:41.318170 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:41.318173 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:41.318189 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:43:41.318367 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:41.318374 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.319795 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.319799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:41.320896 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:41.320908 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.321980 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.321986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:41.322126 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:41.322185 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:43:41.323317 (+ 1132us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:43:41.323390 (+ 73us) client_negotiation.cc:769] Sending connection context
0809 04:43:41.323528 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 04:43:41.323819 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 04:43:41.325060 15619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:41.317367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39180 (local address 127.12.234.3:40143)
0809 04:43:41.317611 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:41.317618 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:41.317641 (+ 23us) server_negotiation.cc:407] Connection header received
0809 04:43:41.317860 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:41.317866 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:41.317981 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:41.318148 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:43:41.318502 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.319649 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.321061 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.321820 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.321951 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:41.322389 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:43:41.323145 (+ 756us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:43:41.323293 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:41.323796 (+ 503us) server_negotiation.cc:299] Negotiation successful
0809 04:43:41.323893 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260809 04:43:41.335456 15406 raft_consensus.cc:1261] T afb20966d6b440b186199f0362061263 P 32ac27385c334ce58e132403a2ef9711 [term 1 FOLLOWER]: Refusing update from remote peer 477443567ac448b4a2595113d452ef22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:43:41.335602 15262 raft_consensus.cc:1261] T afb20966d6b440b186199f0362061263 P 646783dc4be145aab61bcb438398fa88 [term 1 FOLLOWER]: Refusing update from remote peer 477443567ac448b4a2595113d452ef22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:43:41.336580 15631 consensus_queue.cc:1035] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:43:41.336967 15634 consensus_queue.cc:1035] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:43:41.352694 15632 tablet_replica.cc:439] T afb20966d6b440b186199f0362061263 P 32ac27385c334ce58e132403a2ef9711: TxnStatusTablet state changed. Reason: New leader 477443567ac448b4a2595113d452ef22. Latest consensus state: current_term: 1 leader_uuid: "477443567ac448b4a2595113d452ef22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } } }
I20260809 04:43:41.353030 15632 tablet_replica.cc:442] T afb20966d6b440b186199f0362061263 P 32ac27385c334ce58e132403a2ef9711: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:43:41.357687 15631 tablet_replica.cc:439] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "477443567ac448b4a2595113d452ef22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } } }
I20260809 04:43:41.358053 15631 tablet_replica.cc:442] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:43:41.359490 15632 tablet_replica.cc:439] T afb20966d6b440b186199f0362061263 P 32ac27385c334ce58e132403a2ef9711: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "477443567ac448b4a2595113d452ef22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } } }
I20260809 04:43:41.359745 15632 tablet_replica.cc:442] T afb20966d6b440b186199f0362061263 P 32ac27385c334ce58e132403a2ef9711: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:43:41.362469 15651 mvcc.cc:205] Tried to move back new op lower bound from 7316482544979046400 to 7316482544822980608. Current Snapshot: MvccSnapshot[applied={T|T < 7316482544979046400 or (T in {7316482544979046400})}]
I20260809 04:43:41.366361 15630 tablet_replica.cc:439] T afb20966d6b440b186199f0362061263 P 646783dc4be145aab61bcb438398fa88: TxnStatusTablet state changed. Reason: New leader 477443567ac448b4a2595113d452ef22. Latest consensus state: current_term: 1 leader_uuid: "477443567ac448b4a2595113d452ef22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } } }
I20260809 04:43:41.366719 15630 tablet_replica.cc:442] T afb20966d6b440b186199f0362061263 P 646783dc4be145aab61bcb438398fa88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:43:41.370771 15634 tablet_replica.cc:439] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "477443567ac448b4a2595113d452ef22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } } }
I20260809 04:43:41.371062 15634 tablet_replica.cc:442] T afb20966d6b440b186199f0362061263 P 477443567ac448b4a2595113d452ef22: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:43:41.373230 15619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:41.366242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39182 (local address 127.12.234.3:40143)
0809 04:43:41.366430 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:41.366442 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:41.366470 (+ 28us) server_negotiation.cc:407] Connection header received
0809 04:43:41.366568 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:41.366572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:41.366662 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:41.366771 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:43:41.367348 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.368568 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.370402 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.371193 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.371394 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:41.371896 (+ 502us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:43:41.372164 (+ 268us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:43:41.372268 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:41.372815 (+ 547us) server_negotiation.cc:299] Negotiation successful
0809 04:43:41.372905 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 04:43:41.388759 15630 tablet_replica.cc:439] T afb20966d6b440b186199f0362061263 P 646783dc4be145aab61bcb438398fa88: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "477443567ac448b4a2595113d452ef22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } } }
I20260809 04:43:41.389093 15630 tablet_replica.cc:442] T afb20966d6b440b186199f0362061263 P 646783dc4be145aab61bcb438398fa88: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:43:41.401583 15619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:41.395753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:37875 (local address 127.12.234.3:40143)
0809 04:43:41.395912 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:41.395917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:41.396459 (+ 542us) server_negotiation.cc:407] Connection header received
0809 04:43:41.396680 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:41.396685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:41.396752 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:41.396848 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:43:41.397300 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.398091 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.399485 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.399967 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.400180 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:41.400784 (+ 604us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:43:41.400902 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:43:41.401010 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:41.401150 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 04:43:41.401226 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":52}
I20260809 04:43:41.401700 15662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:41.395672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:40143 (local address 127.12.234.3:37875)
0809 04:43:41.396352 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:41.396369 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:41.396547 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:41.396859 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:41.396863 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:41.396873 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:43:41.397152 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:41.397161 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.398228 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.398235 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:41.399322 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:41.399337 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.400194 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.400200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:41.400463 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:41.400518 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:43:41.401007 (+ 489us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:43:41.401026 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 04:43:41.401132 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 04:43:41.401262 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":76,"thread_start_us":355,"threads_started":1}
I20260809 04:43:41.412169 15262 raft_consensus.cc:1261] T 6642ce2983574784a92d4554c329493b P 646783dc4be145aab61bcb438398fa88 [term 1 FOLLOWER]: Refusing update from remote peer 477443567ac448b4a2595113d452ef22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:43:41.412236 15406 raft_consensus.cc:1261] T 6642ce2983574784a92d4554c329493b P 32ac27385c334ce58e132403a2ef9711 [term 1 FOLLOWER]: Refusing update from remote peer 477443567ac448b4a2595113d452ef22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:43:41.412740 15631 consensus_queue.cc:1035] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:43:41.413020 15634 consensus_queue.cc:1035] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:43:41.420881 15650 mvcc.cc:205] Tried to move back new op lower bound from 7316482545299398656 to 7316482544241274880. Current Snapshot: MvccSnapshot[applied={T|T < 7316482545299398656}]
I20260809 04:43:41.492925 15100 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:43:41.493217 15100 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:43:41.496939 15100 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e2763f4b36424b518849a07d07a5f916: Sending DeleteTablet for 3 replicas of tablet 6642ce2983574784a92d4554c329493b
I20260809 04:43:41.498108 15529 tablet_service.cc:1514] Processing DeleteTablet for tablet 6642ce2983574784a92d4554c329493b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:43:41 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:45510
I20260809 04:43:41.498199 15384 tablet_service.cc:1514] Processing DeleteTablet for tablet 6642ce2983574784a92d4554c329493b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:43:41 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:50914
I20260809 04:43:41.498240 15240 tablet_service.cc:1514] Processing DeleteTablet for tablet 6642ce2983574784a92d4554c329493b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:43:41 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:48502
I20260809 04:43:41.502597 15100 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38632:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:43:41.503382 15100 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:43:41.514580 15666 tablet_replica.cc:330] T 6642ce2983574784a92d4554c329493b P 32ac27385c334ce58e132403a2ef9711: stopping tablet replica
I20260809 04:43:41.515127 15666 raft_consensus.cc:2229] T 6642ce2983574784a92d4554c329493b P 32ac27385c334ce58e132403a2ef9711 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:43:41.516204 15239 tablet_service.cc:1467] Processing CreateTablet for tablet 38f7eeb22423482e9ec3699075f121ec (DEFAULT_TABLE table=test-table [id=719205870f394e868d8b477e851619d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:43:41.516832 15667 tablet_replica.cc:330] T 6642ce2983574784a92d4554c329493b P 646783dc4be145aab61bcb438398fa88: stopping tablet replica
I20260809 04:43:41.516913 15239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38f7eeb22423482e9ec3699075f121ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:41.518040 15668 tablet_replica.cc:330] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22: stopping tablet replica
I20260809 04:43:41.518587 15668 raft_consensus.cc:2229] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:43:41.517297 15667 raft_consensus.cc:2229] T 6642ce2983574784a92d4554c329493b P 646783dc4be145aab61bcb438398fa88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:43:41.516499 15383 tablet_service.cc:1467] Processing CreateTablet for tablet 38f7eeb22423482e9ec3699075f121ec (DEFAULT_TABLE table=test-table [id=719205870f394e868d8b477e851619d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:43:41.519786 15383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38f7eeb22423482e9ec3699075f121ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:41.519835 15667 raft_consensus.cc:2258] T 6642ce2983574784a92d4554c329493b P 646783dc4be145aab61bcb438398fa88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:43:41.515596 15666 raft_consensus.cc:2258] T 6642ce2983574784a92d4554c329493b P 32ac27385c334ce58e132403a2ef9711 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:43:41.522972 15667 ts_tablet_manager.cc:1905] T 6642ce2983574784a92d4554c329493b P 646783dc4be145aab61bcb438398fa88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:43:41.524756 15626 tablet_bootstrap.cc:492] T 38f7eeb22423482e9ec3699075f121ec P 646783dc4be145aab61bcb438398fa88: Bootstrap starting.
I20260809 04:43:41.526659 15626 tablet_bootstrap.cc:654] T 38f7eeb22423482e9ec3699075f121ec P 646783dc4be145aab61bcb438398fa88: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:41.527491 15625 tablet_bootstrap.cc:492] T 38f7eeb22423482e9ec3699075f121ec P 32ac27385c334ce58e132403a2ef9711: Bootstrap starting.
I20260809 04:43:41.529882 15625 tablet_bootstrap.cc:654] T 38f7eeb22423482e9ec3699075f121ec P 32ac27385c334ce58e132403a2ef9711: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:41.530505 15666 ts_tablet_manager.cc:1905] T 6642ce2983574784a92d4554c329493b P 32ac27385c334ce58e132403a2ef9711: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:43:41.531776 15528 tablet_service.cc:1467] Processing CreateTablet for tablet 38f7eeb22423482e9ec3699075f121ec (DEFAULT_TABLE table=test-table [id=719205870f394e868d8b477e851619d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:43:41.532543 15528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38f7eeb22423482e9ec3699075f121ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:41.536687 15667 ts_tablet_manager.cc:1918] T 6642ce2983574784a92d4554c329493b P 646783dc4be145aab61bcb438398fa88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:43:41.536923 15667 log.cc:1196] T 6642ce2983574784a92d4554c329493b P 646783dc4be145aab61bcb438398fa88: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-0/wal/wals/6642ce2983574784a92d4554c329493b
I20260809 04:43:41.537607 15667 ts_tablet_manager.cc:1939] T 6642ce2983574784a92d4554c329493b P 646783dc4be145aab61bcb438398fa88: Deleting consensus metadata
I20260809 04:43:41.538036 15625 tablet_bootstrap.cc:492] T 38f7eeb22423482e9ec3699075f121ec P 32ac27385c334ce58e132403a2ef9711: No bootstrap required, opened a new log
I20260809 04:43:41.538229 15625 ts_tablet_manager.cc:1397] T 38f7eeb22423482e9ec3699075f121ec P 32ac27385c334ce58e132403a2ef9711: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.004s
I20260809 04:43:41.538280 15626 tablet_bootstrap.cc:492] T 38f7eeb22423482e9ec3699075f121ec P 646783dc4be145aab61bcb438398fa88: No bootstrap required, opened a new log
I20260809 04:43:41.538450 15626 ts_tablet_manager.cc:1397] T 38f7eeb22423482e9ec3699075f121ec P 646783dc4be145aab61bcb438398fa88: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.008s
I20260809 04:43:41.539067 15625 raft_consensus.cc:348] T 38f7eeb22423482e9ec3699075f121ec P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:41.539173 15626 raft_consensus.cc:348] T 38f7eeb22423482e9ec3699075f121ec P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:41.539342 15625 raft_consensus.cc:374] T 38f7eeb22423482e9ec3699075f121ec P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:41.539419 15626 raft_consensus.cc:374] T 38f7eeb22423482e9ec3699075f121ec P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:41.539450 15625 raft_consensus.cc:729] T 38f7eeb22423482e9ec3699075f121ec P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32ac27385c334ce58e132403a2ef9711, State: Initialized, Role: FOLLOWER
I20260809 04:43:41.532383 15668 raft_consensus.cc:2258] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:43:41.539520 15626 raft_consensus.cc:729] T 38f7eeb22423482e9ec3699075f121ec P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 646783dc4be145aab61bcb438398fa88, State: Initialized, Role: FOLLOWER
I20260809 04:43:41.539719 15625 consensus_queue.cc:260] T 38f7eeb22423482e9ec3699075f121ec P 32ac27385c334ce58e132403a2ef9711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:41.539844 15087 catalog_manager.cc:4841] TS 646783dc4be145aab61bcb438398fa88 (127.12.234.1:43177): tablet 6642ce2983574784a92d4554c329493b (table test-table [id=72de9e6b2a4f4c8ebb40365c998c2523]) successfully deleted
I20260809 04:43:41.539779 15626 consensus_queue.cc:260] T 38f7eeb22423482e9ec3699075f121ec P 646783dc4be145aab61bcb438398fa88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:41.540781 15626 ts_tablet_manager.cc:1428] T 38f7eeb22423482e9ec3699075f121ec P 646783dc4be145aab61bcb438398fa88: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 04:43:41.541323 15624 tablet_bootstrap.cc:492] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22: Bootstrap starting.
I20260809 04:43:41.541839 15625 ts_tablet_manager.cc:1428] T 38f7eeb22423482e9ec3699075f121ec P 32ac27385c334ce58e132403a2ef9711: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260809 04:43:41.543370 15624 tablet_bootstrap.cc:654] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:41.544579 15668 ts_tablet_manager.cc:1905] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:43:41.551726 15666 ts_tablet_manager.cc:1918] T 6642ce2983574784a92d4554c329493b P 32ac27385c334ce58e132403a2ef9711: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:43:41.552014 15666 log.cc:1196] T 6642ce2983574784a92d4554c329493b P 32ac27385c334ce58e132403a2ef9711: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-1/wal/wals/6642ce2983574784a92d4554c329493b
I20260809 04:43:41.552699 15666 ts_tablet_manager.cc:1939] T 6642ce2983574784a92d4554c329493b P 32ac27385c334ce58e132403a2ef9711: Deleting consensus metadata
I20260809 04:43:41.554704 15084 catalog_manager.cc:4841] TS 32ac27385c334ce58e132403a2ef9711 (127.12.234.2:46829): tablet 6642ce2983574784a92d4554c329493b (table test-table [id=72de9e6b2a4f4c8ebb40365c998c2523]) successfully deleted
I20260809 04:43:41.555045 15624 tablet_bootstrap.cc:492] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22: No bootstrap required, opened a new log
I20260809 04:43:41.555300 15624 ts_tablet_manager.cc:1397] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.004s
I20260809 04:43:41.556357 15624 raft_consensus.cc:348] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:41.556682 15624 raft_consensus.cc:374] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:41.556792 15624 raft_consensus.cc:729] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 477443567ac448b4a2595113d452ef22, State: Initialized, Role: FOLLOWER
I20260809 04:43:41.557137 15624 consensus_queue.cc:260] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:41.558131 15668 ts_tablet_manager.cc:1918] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:43:41.558357 15668 log.cc:1196] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-2/wal/wals/6642ce2983574784a92d4554c329493b
I20260809 04:43:41.558697 15624 ts_tablet_manager.cc:1428] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:43:41.559438 15668 ts_tablet_manager.cc:1939] T 6642ce2983574784a92d4554c329493b P 477443567ac448b4a2595113d452ef22: Deleting consensus metadata
I20260809 04:43:41.561327 15085 catalog_manager.cc:4841] TS 477443567ac448b4a2595113d452ef22 (127.12.234.3:40143): tablet 6642ce2983574784a92d4554c329493b (table test-table [id=72de9e6b2a4f4c8ebb40365c998c2523]) successfully deleted
I20260809 04:43:41.811710 15630 raft_consensus.cc:482] T 38f7eeb22423482e9ec3699075f121ec P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:43:41.812011 15630 raft_consensus.cc:504] T 38f7eeb22423482e9ec3699075f121ec P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:41.814069 15630 leader_election.cc:288] T 38f7eeb22423482e9ec3699075f121ec P 646783dc4be145aab61bcb438398fa88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32ac27385c334ce58e132403a2ef9711 (127.12.234.2:46829), 477443567ac448b4a2595113d452ef22 (127.12.234.3:40143)
I20260809 04:43:41.830468 15634 raft_consensus.cc:482] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:43:41.830749 15634 raft_consensus.cc:504] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:41.831712 15634 leader_election.cc:288] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 646783dc4be145aab61bcb438398fa88 (127.12.234.1:43177), 32ac27385c334ce58e132403a2ef9711 (127.12.234.2:46829)
I20260809 04:43:41.833492 15406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38f7eeb22423482e9ec3699075f121ec" candidate_uuid: "477443567ac448b4a2595113d452ef22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32ac27385c334ce58e132403a2ef9711" is_pre_election: true
I20260809 04:43:41.833833 15406 raft_consensus.cc:2454] T 38f7eeb22423482e9ec3699075f121ec P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 477443567ac448b4a2595113d452ef22 in term 0.
I20260809 04:43:41.833101 15261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38f7eeb22423482e9ec3699075f121ec" candidate_uuid: "477443567ac448b4a2595113d452ef22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "646783dc4be145aab61bcb438398fa88" is_pre_election: true
I20260809 04:43:41.834121 15261 raft_consensus.cc:2454] T 38f7eeb22423482e9ec3699075f121ec P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 477443567ac448b4a2595113d452ef22 in term 0.
I20260809 04:43:41.834466 15477 leader_election.cc:302] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32ac27385c334ce58e132403a2ef9711, 477443567ac448b4a2595113d452ef22; no voters:
I20260809 04:43:41.835278 15634 raft_consensus.cc:2781] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:43:41.835464 15634 raft_consensus.cc:482] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:43:41.835587 15634 raft_consensus.cc:3037] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:41.839465 15672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:41.814306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:46829 (local address 127.12.234.1:47641)
0809 04:43:41.829875 (+ 15569us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:41.829895 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:41.830035 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:41.830458 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:41.830463 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:41.830494 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:41.830891 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:41.830909 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.832633 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.832639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:41.836270 (+ 3631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:41.836291 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.838450 (+ 2159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.838459 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:41.838671 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:41.838726 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 04:43:41.838841 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 04:43:41.839037 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15366,"mutex_wait_us":26,"thread_start_us":15079,"threads_started":1}
I20260809 04:43:41.839457 15634 raft_consensus.cc:504] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:41.840415 15674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:41.814472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:47641 (local address 127.12.234.2:46829)
0809 04:43:41.829420 (+ 14948us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:41.829427 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:41.829988 (+ 561us) server_negotiation.cc:407] Connection header received
0809 04:43:41.830222 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:41.830228 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:41.830321 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:41.830436 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:41.831132 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.832418 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.836524 (+ 4106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.837985 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.838193 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:41.838282 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:41.839728 (+ 1446us) server_negotiation.cc:299] Negotiation successful
0809 04:43:41.839917 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14814,"thread_start_us":14703,"threads_started":1}
I20260809 04:43:41.840955 15261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38f7eeb22423482e9ec3699075f121ec" candidate_uuid: "477443567ac448b4a2595113d452ef22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "646783dc4be145aab61bcb438398fa88"
I20260809 04:43:41.841177 15261 raft_consensus.cc:3037] T 38f7eeb22423482e9ec3699075f121ec P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:41.841483 15619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:41.814962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:57825 (local address 127.12.234.3:40143)
0809 04:43:41.815222 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:41.815231 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:41.828889 (+ 13658us) server_negotiation.cc:407] Connection header received
0809 04:43:41.829984 (+ 1095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:41.829990 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:41.830106 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:41.830229 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:41.831747 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.833116 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.837627 (+ 4511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.839399 (+ 1772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.839548 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:41.839626 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:41.840943 (+ 1317us) server_negotiation.cc:299] Negotiation successful
0809 04:43:41.841098 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 04:43:41.842104 15406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38f7eeb22423482e9ec3699075f121ec" candidate_uuid: "646783dc4be145aab61bcb438398fa88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32ac27385c334ce58e132403a2ef9711" is_pre_election: true
I20260809 04:43:41.842360 15406 raft_consensus.cc:2454] T 38f7eeb22423482e9ec3699075f121ec P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 646783dc4be145aab61bcb438398fa88 in term 0.
I20260809 04:43:41.842483 15673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:41.814306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:40143 (local address 127.12.234.1:57825)
0809 04:43:41.828694 (+ 14388us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:41.828728 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:41.829753 (+ 1025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:41.830880 (+ 1127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:41.830885 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:41.830926 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:41.831487 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:41.831502 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.833417 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.833422 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:41.837407 (+ 3985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:41.837426 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:41.840293 (+ 2867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:41.840300 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:41.840684 (+ 384us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:41.840731 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 04:43:41.841713 (+ 982us) client_negotiation.cc:240] Negotiation successful
0809 04:43:41.841926 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14168,"thread_start_us":14834,"threads_started":1}
I20260809 04:43:41.843119 15634 leader_election.cc:288] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22 [CANDIDATE]: Term 1 election: Requested vote from peers 646783dc4be145aab61bcb438398fa88 (127.12.234.1:43177), 32ac27385c334ce58e132403a2ef9711 (127.12.234.2:46829)
I20260809 04:43:41.843179 15189 leader_election.cc:302] T 38f7eeb22423482e9ec3699075f121ec P 646783dc4be145aab61bcb438398fa88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32ac27385c334ce58e132403a2ef9711, 646783dc4be145aab61bcb438398fa88; no voters:
I20260809 04:43:41.844300 15406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38f7eeb22423482e9ec3699075f121ec" candidate_uuid: "477443567ac448b4a2595113d452ef22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32ac27385c334ce58e132403a2ef9711"
I20260809 04:43:41.844539 15406 raft_consensus.cc:3037] T 38f7eeb22423482e9ec3699075f121ec P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:41.844559 15261 raft_consensus.cc:2454] T 38f7eeb22423482e9ec3699075f121ec P 646783dc4be145aab61bcb438398fa88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 477443567ac448b4a2595113d452ef22 in term 1.
I20260809 04:43:41.844436 15551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38f7eeb22423482e9ec3699075f121ec" candidate_uuid: "646783dc4be145aab61bcb438398fa88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "477443567ac448b4a2595113d452ef22" is_pre_election: true
I20260809 04:43:41.844698 15630 raft_consensus.cc:2741] T 38f7eeb22423482e9ec3699075f121ec P 646783dc4be145aab61bcb438398fa88 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260809 04:43:41.845141 15551 raft_consensus.cc:2379] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 646783dc4be145aab61bcb438398fa88 in current term 1: Already voted for candidate 477443567ac448b4a2595113d452ef22 in this term.
I20260809 04:43:41.845714 15480 leader_election.cc:302] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 477443567ac448b4a2595113d452ef22, 646783dc4be145aab61bcb438398fa88; no voters:
I20260809 04:43:41.846227 15634 raft_consensus.cc:2781] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:43:41.846760 15634 raft_consensus.cc:686] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22 [term 1 LEADER]: Becoming Leader. State: Replica: 477443567ac448b4a2595113d452ef22, State: Running, Role: LEADER
I20260809 04:43:41.847131 15634 consensus_queue.cc:237] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:41.848299 15406 raft_consensus.cc:2454] T 38f7eeb22423482e9ec3699075f121ec P 32ac27385c334ce58e132403a2ef9711 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 477443567ac448b4a2595113d452ef22 in term 1.
I20260809 04:43:41.850272 15100 catalog_manager.cc:5495] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22 reported cstate change: term changed from 0 to 1, leader changed from <none> to 477443567ac448b4a2595113d452ef22 (127.12.234.3). New cstate: current_term: 1 leader_uuid: "477443567ac448b4a2595113d452ef22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } health_report { overall_health: HEALTHY } } }
I20260809 04:43:41.917119 15406 raft_consensus.cc:1261] T 38f7eeb22423482e9ec3699075f121ec P 32ac27385c334ce58e132403a2ef9711 [term 1 FOLLOWER]: Refusing update from remote peer 477443567ac448b4a2595113d452ef22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:43:41.917239 15261 raft_consensus.cc:1261] T 38f7eeb22423482e9ec3699075f121ec P 646783dc4be145aab61bcb438398fa88 [term 1 FOLLOWER]: Refusing update from remote peer 477443567ac448b4a2595113d452ef22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:43:41.917728 15631 consensus_queue.cc:1035] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:43:41.918188 15631 consensus_queue.cc:1035] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 04:43:41.952854 15650 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 04:43:43.605288 15101 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:43:43.605535 15101 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:43:43.610209 15101 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e2763f4b36424b518849a07d07a5f916: Sending DeleteTablet for 3 replicas of tablet 38f7eeb22423482e9ec3699075f121ec
I20260809 04:43:43.611212 15383 tablet_service.cc:1514] Processing DeleteTablet for tablet 38f7eeb22423482e9ec3699075f121ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:43:43 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:50914
I20260809 04:43:43.611265 15239 tablet_service.cc:1514] Processing DeleteTablet for tablet 38f7eeb22423482e9ec3699075f121ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:43:43 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:48502
I20260809 04:43:43.611371 15528 tablet_service.cc:1514] Processing DeleteTablet for tablet 38f7eeb22423482e9ec3699075f121ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:43:43 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:45510
I20260809 04:43:43.624346 15706 tablet_replica.cc:330] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22: stopping tablet replica
I20260809 04:43:43.624516 15705 tablet_replica.cc:330] T 38f7eeb22423482e9ec3699075f121ec P 646783dc4be145aab61bcb438398fa88: stopping tablet replica
I20260809 04:43:43.624701 15706 raft_consensus.cc:2229] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:43:43.624750 15705 raft_consensus.cc:2229] T 38f7eeb22423482e9ec3699075f121ec P 646783dc4be145aab61bcb438398fa88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:43:43.625022 15705 raft_consensus.cc:2258] T 38f7eeb22423482e9ec3699075f121ec P 646783dc4be145aab61bcb438398fa88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:43:43.625239 15706 raft_consensus.cc:2258] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:43:43.627254 15705 ts_tablet_manager.cc:1905] T 38f7eeb22423482e9ec3699075f121ec P 646783dc4be145aab61bcb438398fa88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:43:43.627631 15706 ts_tablet_manager.cc:1905] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:43:43.633711 15703 tablet_replica.cc:330] T 38f7eeb22423482e9ec3699075f121ec P 32ac27385c334ce58e132403a2ef9711: stopping tablet replica
I20260809 04:43:43.634038 15703 raft_consensus.cc:2229] T 38f7eeb22423482e9ec3699075f121ec P 32ac27385c334ce58e132403a2ef9711 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:43:43.634372 15703 raft_consensus.cc:2258] T 38f7eeb22423482e9ec3699075f121ec P 32ac27385c334ce58e132403a2ef9711 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:43:43.635635 15703 ts_tablet_manager.cc:1905] T 38f7eeb22423482e9ec3699075f121ec P 32ac27385c334ce58e132403a2ef9711: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 04:43:43 dist-test-slave-1jjb krb5kdc[15054](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250623, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
I20260809 04:43:43.687808 15703 ts_tablet_manager.cc:1918] T 38f7eeb22423482e9ec3699075f121ec P 32ac27385c334ce58e132403a2ef9711: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 04:43:43.688057 15703 log.cc:1196] T 38f7eeb22423482e9ec3699075f121ec P 32ac27385c334ce58e132403a2ef9711: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-1/wal/wals/38f7eeb22423482e9ec3699075f121ec
I20260809 04:43:43.688025 15706 ts_tablet_manager.cc:1918] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 04:43:43.688200 15706 log.cc:1196] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-2/wal/wals/38f7eeb22423482e9ec3699075f121ec
I20260809 04:43:43.688639 15703 ts_tablet_manager.cc:1939] T 38f7eeb22423482e9ec3699075f121ec P 32ac27385c334ce58e132403a2ef9711: Deleting consensus metadata
I20260809 04:43:43.688707 15706 ts_tablet_manager.cc:1939] T 38f7eeb22423482e9ec3699075f121ec P 477443567ac448b4a2595113d452ef22: Deleting consensus metadata
I20260809 04:43:43.688704 15705 ts_tablet_manager.cc:1918] T 38f7eeb22423482e9ec3699075f121ec P 646783dc4be145aab61bcb438398fa88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 04:43:43.688869 15705 log.cc:1196] T 38f7eeb22423482e9ec3699075f121ec P 646783dc4be145aab61bcb438398fa88: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-0/wal/wals/38f7eeb22423482e9ec3699075f121ec
I20260809 04:43:43.689435 15705 ts_tablet_manager.cc:1939] T 38f7eeb22423482e9ec3699075f121ec P 646783dc4be145aab61bcb438398fa88: Deleting consensus metadata
Aug 09 04:43:43 dist-test-slave-1jjb krb5kdc[15054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250623, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:43.708735 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:43.692190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44440 (local address 127.12.234.62:42671)
0809 04:43:43.692926 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:43.692930 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:43.693037 (+ 107us) server_negotiation.cc:407] Connection header received
0809 04:43:43.693215 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:43.693219 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:43.693289 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:43.693390 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:43.695018 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:43.696024 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:43.697545 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:43.698094 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:43.698211 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:43.702414 (+ 4203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:43.702420 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:43.702423 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:43.702459 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:43.706228 (+ 3769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:43.706924 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:43.706928 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:43.706930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:43.707065 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:43.707370 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:43.707373 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:43.707375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:43.707690 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:43.707886 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:43.708219 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 04:43:43.708404 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":452,"threads_started":1}
I20260809 04:43:43.712697 15101 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44440:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:43:43.713312 15101 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:43:43.720553 15239 tablet_service.cc:1467] Processing CreateTablet for tablet 94bdebb06c8944239f964b9148a31b83 (DEFAULT_TABLE table=test-table [id=79c5cb988bc24fb7803052c05c848a6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:43:43.720717 15383 tablet_service.cc:1467] Processing CreateTablet for tablet 94bdebb06c8944239f964b9148a31b83 (DEFAULT_TABLE table=test-table [id=79c5cb988bc24fb7803052c05c848a6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:43:43.720894 15528 tablet_service.cc:1467] Processing CreateTablet for tablet 94bdebb06c8944239f964b9148a31b83 (DEFAULT_TABLE table=test-table [id=79c5cb988bc24fb7803052c05c848a6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:43:43.721143 15239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94bdebb06c8944239f964b9148a31b83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:43.721249 15383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94bdebb06c8944239f964b9148a31b83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:43.721467 15528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94bdebb06c8944239f964b9148a31b83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:43.727348 15715 tablet_bootstrap.cc:492] T 94bdebb06c8944239f964b9148a31b83 P 32ac27385c334ce58e132403a2ef9711: Bootstrap starting.
I20260809 04:43:43.728595 15716 tablet_bootstrap.cc:492] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88: Bootstrap starting.
I20260809 04:43:43.729400 15715 tablet_bootstrap.cc:654] T 94bdebb06c8944239f964b9148a31b83 P 32ac27385c334ce58e132403a2ef9711: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:43.730520 15717 tablet_bootstrap.cc:492] T 94bdebb06c8944239f964b9148a31b83 P 477443567ac448b4a2595113d452ef22: Bootstrap starting.
I20260809 04:43:43.730811 15716 tablet_bootstrap.cc:654] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:43.732843 15717 tablet_bootstrap.cc:654] T 94bdebb06c8944239f964b9148a31b83 P 477443567ac448b4a2595113d452ef22: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:43.733716 15716 tablet_bootstrap.cc:492] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88: No bootstrap required, opened a new log
I20260809 04:43:43.733882 15716 ts_tablet_manager.cc:1397] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20260809 04:43:43.735199 15716 raft_consensus.cc:348] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:43.735476 15716 raft_consensus.cc:374] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:43.735628 15716 raft_consensus.cc:729] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 646783dc4be145aab61bcb438398fa88, State: Initialized, Role: FOLLOWER
I20260809 04:43:43.735836 15715 tablet_bootstrap.cc:492] T 94bdebb06c8944239f964b9148a31b83 P 32ac27385c334ce58e132403a2ef9711: No bootstrap required, opened a new log
I20260809 04:43:43.735896 15716 consensus_queue.cc:260] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:43.736169 15715 ts_tablet_manager.cc:1397] T 94bdebb06c8944239f964b9148a31b83 P 32ac27385c334ce58e132403a2ef9711: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260809 04:43:43.736241 15717 tablet_bootstrap.cc:492] T 94bdebb06c8944239f964b9148a31b83 P 477443567ac448b4a2595113d452ef22: No bootstrap required, opened a new log
I20260809 04:43:43.736487 15717 ts_tablet_manager.cc:1397] T 94bdebb06c8944239f964b9148a31b83 P 477443567ac448b4a2595113d452ef22: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 04:43:43.737332 15715 raft_consensus.cc:348] T 94bdebb06c8944239f964b9148a31b83 P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:43.737571 15715 raft_consensus.cc:374] T 94bdebb06c8944239f964b9148a31b83 P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:43.737666 15715 raft_consensus.cc:729] T 94bdebb06c8944239f964b9148a31b83 P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32ac27385c334ce58e132403a2ef9711, State: Initialized, Role: FOLLOWER
I20260809 04:43:43.737912 15715 consensus_queue.cc:260] T 94bdebb06c8944239f964b9148a31b83 P 32ac27385c334ce58e132403a2ef9711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:43.737933 15717 raft_consensus.cc:348] T 94bdebb06c8944239f964b9148a31b83 P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:43.738283 15717 raft_consensus.cc:374] T 94bdebb06c8944239f964b9148a31b83 P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:43.738448 15717 raft_consensus.cc:729] T 94bdebb06c8944239f964b9148a31b83 P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 477443567ac448b4a2595113d452ef22, State: Initialized, Role: FOLLOWER
I20260809 04:43:43.740270 15717 consensus_queue.cc:260] T 94bdebb06c8944239f964b9148a31b83 P 477443567ac448b4a2595113d452ef22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:43.741355 15717 ts_tablet_manager.cc:1428] T 94bdebb06c8944239f964b9148a31b83 P 477443567ac448b4a2595113d452ef22: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260809 04:43:43.746420 15716 ts_tablet_manager.cc:1428] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88: Time spent starting tablet: real 0.012s user 0.007s sys 0.002s
I20260809 04:43:43.747270 15715 ts_tablet_manager.cc:1428] T 94bdebb06c8944239f964b9148a31b83 P 32ac27385c334ce58e132403a2ef9711: Time spent starting tablet: real 0.011s user 0.007s sys 0.003s
I20260809 04:43:43.759737 15084 catalog_manager.cc:4841] TS 32ac27385c334ce58e132403a2ef9711 (127.12.234.2:46829): tablet 38f7eeb22423482e9ec3699075f121ec (table test-table [id=719205870f394e868d8b477e851619d3]) successfully deleted
I20260809 04:43:43.771924 15721 raft_consensus.cc:482] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:43:43.772279 15721 raft_consensus.cc:504] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:43.773159 15721 leader_election.cc:288] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32ac27385c334ce58e132403a2ef9711 (127.12.234.2:46829), 477443567ac448b4a2595113d452ef22 (127.12.234.3:40143)
I20260809 04:43:43.773805 15406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94bdebb06c8944239f964b9148a31b83" candidate_uuid: "646783dc4be145aab61bcb438398fa88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32ac27385c334ce58e132403a2ef9711" is_pre_election: true
I20260809 04:43:43.773850 15551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94bdebb06c8944239f964b9148a31b83" candidate_uuid: "646783dc4be145aab61bcb438398fa88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "477443567ac448b4a2595113d452ef22" is_pre_election: true
I20260809 04:43:43.774142 15551 raft_consensus.cc:2454] T 94bdebb06c8944239f964b9148a31b83 P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 646783dc4be145aab61bcb438398fa88 in term 0.
I20260809 04:43:43.774194 15406 raft_consensus.cc:2454] T 94bdebb06c8944239f964b9148a31b83 P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 646783dc4be145aab61bcb438398fa88 in term 0.
I20260809 04:43:43.774718 15190 leader_election.cc:302] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 477443567ac448b4a2595113d452ef22, 646783dc4be145aab61bcb438398fa88; no voters:
I20260809 04:43:43.775079 15721 raft_consensus.cc:2781] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:43:43.775218 15721 raft_consensus.cc:482] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:43:43.775327 15721 raft_consensus.cc:3037] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:43.779956 15087 catalog_manager.cc:4841] TS 646783dc4be145aab61bcb438398fa88 (127.12.234.1:43177): tablet 38f7eeb22423482e9ec3699075f121ec (table test-table [id=719205870f394e868d8b477e851619d3]) successfully deleted
I20260809 04:43:43.779956 15085 catalog_manager.cc:4841] TS 477443567ac448b4a2595113d452ef22 (127.12.234.3:40143): tablet 38f7eeb22423482e9ec3699075f121ec (table test-table [id=719205870f394e868d8b477e851619d3]) successfully deleted
I20260809 04:43:43.785137 15721 raft_consensus.cc:504] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:43.786224 15721 leader_election.cc:288] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88 [CANDIDATE]: Term 1 election: Requested vote from peers 32ac27385c334ce58e132403a2ef9711 (127.12.234.2:46829), 477443567ac448b4a2595113d452ef22 (127.12.234.3:40143)
I20260809 04:43:43.786655 15406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94bdebb06c8944239f964b9148a31b83" candidate_uuid: "646783dc4be145aab61bcb438398fa88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32ac27385c334ce58e132403a2ef9711"
I20260809 04:43:43.786896 15551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94bdebb06c8944239f964b9148a31b83" candidate_uuid: "646783dc4be145aab61bcb438398fa88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "477443567ac448b4a2595113d452ef22"
I20260809 04:43:43.786947 15406 raft_consensus.cc:3037] T 94bdebb06c8944239f964b9148a31b83 P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:43.787127 15551 raft_consensus.cc:3037] T 94bdebb06c8944239f964b9148a31b83 P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:43.789860 15551 raft_consensus.cc:2454] T 94bdebb06c8944239f964b9148a31b83 P 477443567ac448b4a2595113d452ef22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 646783dc4be145aab61bcb438398fa88 in term 1.
I20260809 04:43:43.790464 15190 leader_election.cc:302] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 477443567ac448b4a2595113d452ef22, 646783dc4be145aab61bcb438398fa88; no voters:
I20260809 04:43:43.790882 15721 raft_consensus.cc:2781] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:43:43.791575 15721 raft_consensus.cc:686] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88 [term 1 LEADER]: Becoming Leader. State: Replica: 646783dc4be145aab61bcb438398fa88, State: Running, Role: LEADER
I20260809 04:43:43.792115 15721 consensus_queue.cc:237] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:43.795861 15100 catalog_manager.cc:5495] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88 reported cstate change: term changed from 0 to 1, leader changed from <none> to 646783dc4be145aab61bcb438398fa88 (127.12.234.1). New cstate: current_term: 1 leader_uuid: "646783dc4be145aab61bcb438398fa88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } health_report { overall_health: UNKNOWN } } }
I20260809 04:43:43.824893 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:43.816908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45840 (local address 127.12.234.1:43177)
0809 04:43:43.817963 (+ 1055us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:43.817967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:43.817988 (+ 21us) server_negotiation.cc:407] Connection header received
0809 04:43:43.818081 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:43.818089 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:43.818173 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:43.818334 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:43:43.818777 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:43.819742 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:43.821730 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:43.822316 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:43.822430 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:43.823002 (+ 572us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:43:43.823921 (+ 919us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:43:43.824143 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:43.824297 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 04:43:43.824441 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":892,"thread_start_us":675,"threads_started":1}
I20260809 04:43:43.842888 15732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:43.836269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:52115 (local address 127.12.234.3:40143)
0809 04:43:43.837143 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:43.837148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:43.837192 (+ 44us) server_negotiation.cc:407] Connection header received
0809 04:43:43.837259 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:43.837262 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:43.837327 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:43.837414 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:43:43.838033 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:43.839124 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:43.841049 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:43.841584 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:43.841700 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:43.842223 (+ 523us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:43:43.842330 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:43:43.842400 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:43.842547 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 04:43:43.842609 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":714,"thread_start_us":579,"threads_started":1}
I20260809 04:43:43.843006 15729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:43.836130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:40143 (local address 127.12.234.1:52115)
0809 04:43:43.837035 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:43.837054 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:43.837172 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:43.837479 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:43.837482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:43.837492 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:43:43.837847 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:43.837856 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:43.839309 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:43.839313 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:43.840889 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:43.840915 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:43.841785 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:43.841790 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:43.841976 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:43.842046 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:43:43.842425 (+ 379us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:43:43.842438 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 04:43:43.842506 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 04:43:43.842605 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":573,"threads_started":1}
I20260809 04:43:43.852726 15551 raft_consensus.cc:1261] T 94bdebb06c8944239f964b9148a31b83 P 477443567ac448b4a2595113d452ef22 [term 1 FOLLOWER]: Refusing update from remote peer 646783dc4be145aab61bcb438398fa88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:43:43.853621 15723 consensus_queue.cc:1035] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:43:43.883230 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:43.876434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:60697 (local address 127.12.234.1:43177)
0809 04:43:43.876604 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:43.876610 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:43.877664 (+ 1054us) server_negotiation.cc:407] Connection header received
0809 04:43:43.877815 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:43.877820 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:43.877901 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:43.877998 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:43:43.878499 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:43.879329 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:43.881257 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:43.881718 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:43.881858 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:43.882375 (+ 517us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:43:43.882480 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:43:43.882585 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:43.882816 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 04:43:43.882901 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:43:43.883389 15735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:43.876374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:43177 (local address 127.12.234.3:60697)
0809 04:43:43.877554 (+ 1180us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:43.877573 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:43.877694 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:43.878012 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:43.878015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:43.878025 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:43:43.878308 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:43.878316 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:43.879478 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:43.879486 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:43.881118 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:43.881132 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:43.881918 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:43.881926 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:43.882166 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:43.882205 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:43:43.882605 (+ 400us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:43:43.882617 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 04:43:43.882760 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 04:43:43.882902 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":991,"thread_start_us":876,"threads_started":1}
I20260809 04:43:43.887490 15406 raft_consensus.cc:2454] T 94bdebb06c8944239f964b9148a31b83 P 32ac27385c334ce58e132403a2ef9711 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 646783dc4be145aab61bcb438398fa88 in term 1.
I20260809 04:43:43.888684 15405 raft_consensus.cc:1261] T 94bdebb06c8944239f964b9148a31b83 P 32ac27385c334ce58e132403a2ef9711 [term 1 FOLLOWER]: Refusing update from remote peer 646783dc4be145aab61bcb438398fa88: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:43:43.891479 15721 consensus_queue.cc:1035] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:43:43.928821 15100 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:43:43.929193 15100 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:43:43.933075 15100 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e2763f4b36424b518849a07d07a5f916: Sending DeleteTablet for 3 replicas of tablet 94bdebb06c8944239f964b9148a31b83
I20260809 04:43:43.934024 15239 tablet_service.cc:1514] Processing DeleteTablet for tablet 94bdebb06c8944239f964b9148a31b83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:43:43 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:48502
I20260809 04:43:43.934319 15705 tablet_replica.cc:330] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88: stopping tablet replica
I20260809 04:43:43.934285 15383 tablet_service.cc:1514] Processing DeleteTablet for tablet 94bdebb06c8944239f964b9148a31b83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:43:43 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:50914
I20260809 04:43:43.934571 15705 raft_consensus.cc:2229] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:43:43.934573 15528 tablet_service.cc:1514] Processing DeleteTablet for tablet 94bdebb06c8944239f964b9148a31b83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:43:43 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:45510
I20260809 04:43:43.934736 15703 tablet_replica.cc:330] T 94bdebb06c8944239f964b9148a31b83 P 32ac27385c334ce58e132403a2ef9711: stopping tablet replica
I20260809 04:43:43.935038 15706 tablet_replica.cc:330] T 94bdebb06c8944239f964b9148a31b83 P 477443567ac448b4a2595113d452ef22: stopping tablet replica
I20260809 04:43:43.935016 15703 raft_consensus.cc:2229] T 94bdebb06c8944239f964b9148a31b83 P 32ac27385c334ce58e132403a2ef9711 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:43:43.935223 15705 raft_consensus.cc:2258] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:43:43.935256 15703 raft_consensus.cc:2258] T 94bdebb06c8944239f964b9148a31b83 P 32ac27385c334ce58e132403a2ef9711 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:43:43.935269 15706 raft_consensus.cc:2229] T 94bdebb06c8944239f964b9148a31b83 P 477443567ac448b4a2595113d452ef22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:43:43.935643 15706 raft_consensus.cc:2258] T 94bdebb06c8944239f964b9148a31b83 P 477443567ac448b4a2595113d452ef22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:43:43.936863 15703 ts_tablet_manager.cc:1905] T 94bdebb06c8944239f964b9148a31b83 P 32ac27385c334ce58e132403a2ef9711: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:43:43.937242 15706 ts_tablet_manager.cc:1905] T 94bdebb06c8944239f964b9148a31b83 P 477443567ac448b4a2595113d452ef22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:43:43.938139 15100 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44440:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:43:43.938843 15100 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:43:43.940836 15705 ts_tablet_manager.cc:1905] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:43:43.946997 15239 tablet_service.cc:1467] Processing CreateTablet for tablet 2d1c066df40e4ecd97562731c2ed0ef9 (DEFAULT_TABLE table=test-table [id=52656bb6093546a2872613d7c18b58cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:43:43.947656 15528 tablet_service.cc:1467] Processing CreateTablet for tablet 2d1c066df40e4ecd97562731c2ed0ef9 (DEFAULT_TABLE table=test-table [id=52656bb6093546a2872613d7c18b58cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:43:43.947659 15383 tablet_service.cc:1467] Processing CreateTablet for tablet 2d1c066df40e4ecd97562731c2ed0ef9 (DEFAULT_TABLE table=test-table [id=52656bb6093546a2872613d7c18b58cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:43:43.988688 15383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d1c066df40e4ecd97562731c2ed0ef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:43.988754 15528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d1c066df40e4ecd97562731c2ed0ef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:43.996904 15239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d1c066df40e4ecd97562731c2ed0ef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:43.997457 15715 tablet_bootstrap.cc:492] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711: Bootstrap starting.
I20260809 04:43:43.999691 15715 tablet_bootstrap.cc:654] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:44.004338 15715 tablet_bootstrap.cc:492] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711: No bootstrap required, opened a new log
I20260809 04:43:44.004621 15715 ts_tablet_manager.cc:1397] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.005s
I20260809 04:43:44.005475 15715 raft_consensus.cc:348] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:44.005751 15715 raft_consensus.cc:374] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:44.005854 15715 raft_consensus.cc:729] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32ac27385c334ce58e132403a2ef9711, State: Initialized, Role: FOLLOWER
I20260809 04:43:44.006198 15715 consensus_queue.cc:260] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:44.006978 15715 ts_tablet_manager.cc:1428] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 04:43:44.090987 15706 ts_tablet_manager.cc:1918] T 94bdebb06c8944239f964b9148a31b83 P 477443567ac448b4a2595113d452ef22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:43:44.090986 15703 ts_tablet_manager.cc:1918] T 94bdebb06c8944239f964b9148a31b83 P 32ac27385c334ce58e132403a2ef9711: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:43:44.091208 15703 log.cc:1196] T 94bdebb06c8944239f964b9148a31b83 P 32ac27385c334ce58e132403a2ef9711: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-1/wal/wals/94bdebb06c8944239f964b9148a31b83
I20260809 04:43:44.091229 15706 log.cc:1196] T 94bdebb06c8944239f964b9148a31b83 P 477443567ac448b4a2595113d452ef22: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-2/wal/wals/94bdebb06c8944239f964b9148a31b83
I20260809 04:43:44.091711 15706 ts_tablet_manager.cc:1939] T 94bdebb06c8944239f964b9148a31b83 P 477443567ac448b4a2595113d452ef22: Deleting consensus metadata
I20260809 04:43:44.091715 15703 ts_tablet_manager.cc:1939] T 94bdebb06c8944239f964b9148a31b83 P 32ac27385c334ce58e132403a2ef9711: Deleting consensus metadata
I20260809 04:43:44.095742 15085 catalog_manager.cc:4841] TS 477443567ac448b4a2595113d452ef22 (127.12.234.3:40143): tablet 94bdebb06c8944239f964b9148a31b83 (table test-table [id=79c5cb988bc24fb7803052c05c848a6c]) successfully deleted
I20260809 04:43:44.096467 15716 tablet_bootstrap.cc:492] T 2d1c066df40e4ecd97562731c2ed0ef9 P 646783dc4be145aab61bcb438398fa88: Bootstrap starting.
I20260809 04:43:44.097023 15705 ts_tablet_manager.cc:1918] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:43:44.097185 15705 log.cc:1196] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-0/wal/wals/94bdebb06c8944239f964b9148a31b83
I20260809 04:43:44.097824 15705 ts_tablet_manager.cc:1939] T 94bdebb06c8944239f964b9148a31b83 P 646783dc4be145aab61bcb438398fa88: Deleting consensus metadata
I20260809 04:43:44.098804 15716 tablet_bootstrap.cc:654] T 2d1c066df40e4ecd97562731c2ed0ef9 P 646783dc4be145aab61bcb438398fa88: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:44.100211 15717 tablet_bootstrap.cc:492] T 2d1c066df40e4ecd97562731c2ed0ef9 P 477443567ac448b4a2595113d452ef22: Bootstrap starting.
I20260809 04:43:44.101028 15084 catalog_manager.cc:4841] TS 32ac27385c334ce58e132403a2ef9711 (127.12.234.2:46829): tablet 94bdebb06c8944239f964b9148a31b83 (table test-table [id=79c5cb988bc24fb7803052c05c848a6c]) successfully deleted
I20260809 04:43:44.101643 15716 tablet_bootstrap.cc:492] T 2d1c066df40e4ecd97562731c2ed0ef9 P 646783dc4be145aab61bcb438398fa88: No bootstrap required, opened a new log
I20260809 04:43:44.101890 15716 ts_tablet_manager.cc:1397] T 2d1c066df40e4ecd97562731c2ed0ef9 P 646783dc4be145aab61bcb438398fa88: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260809 04:43:44.102200 15717 tablet_bootstrap.cc:654] T 2d1c066df40e4ecd97562731c2ed0ef9 P 477443567ac448b4a2595113d452ef22: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:44.102841 15716 raft_consensus.cc:348] T 2d1c066df40e4ecd97562731c2ed0ef9 P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:44.103214 15716 raft_consensus.cc:374] T 2d1c066df40e4ecd97562731c2ed0ef9 P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:44.103358 15716 raft_consensus.cc:729] T 2d1c066df40e4ecd97562731c2ed0ef9 P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 646783dc4be145aab61bcb438398fa88, State: Initialized, Role: FOLLOWER
I20260809 04:43:44.103689 15716 consensus_queue.cc:260] T 2d1c066df40e4ecd97562731c2ed0ef9 P 646783dc4be145aab61bcb438398fa88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:44.105095 15716 ts_tablet_manager.cc:1428] T 2d1c066df40e4ecd97562731c2ed0ef9 P 646783dc4be145aab61bcb438398fa88: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:43:44.111970 15717 tablet_bootstrap.cc:492] T 2d1c066df40e4ecd97562731c2ed0ef9 P 477443567ac448b4a2595113d452ef22: No bootstrap required, opened a new log
I20260809 04:43:44.112205 15717 ts_tablet_manager.cc:1397] T 2d1c066df40e4ecd97562731c2ed0ef9 P 477443567ac448b4a2595113d452ef22: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.003s
I20260809 04:43:44.113130 15717 raft_consensus.cc:348] T 2d1c066df40e4ecd97562731c2ed0ef9 P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:44.113420 15717 raft_consensus.cc:374] T 2d1c066df40e4ecd97562731c2ed0ef9 P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:44.113533 15717 raft_consensus.cc:729] T 2d1c066df40e4ecd97562731c2ed0ef9 P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 477443567ac448b4a2595113d452ef22, State: Initialized, Role: FOLLOWER
I20260809 04:43:44.113835 15717 consensus_queue.cc:260] T 2d1c066df40e4ecd97562731c2ed0ef9 P 477443567ac448b4a2595113d452ef22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:44.114621 15717 ts_tablet_manager.cc:1428] T 2d1c066df40e4ecd97562731c2ed0ef9 P 477443567ac448b4a2595113d452ef22: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 04:43:44.118522 15087 catalog_manager.cc:4841] TS 646783dc4be145aab61bcb438398fa88 (127.12.234.1:43177): tablet 94bdebb06c8944239f964b9148a31b83 (table test-table [id=79c5cb988bc24fb7803052c05c848a6c]) successfully deleted
I20260809 04:43:44.131323 15722 raft_consensus.cc:482] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:43:44.131711 15722 raft_consensus.cc:504] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:44.133881 15722 leader_election.cc:288] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 646783dc4be145aab61bcb438398fa88 (127.12.234.1:43177), 477443567ac448b4a2595113d452ef22 (127.12.234.3:40143)
I20260809 04:43:44.144055 15732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:44.134529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:44893 (local address 127.12.234.3:40143)
0809 04:43:44.134706 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:44.134714 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:44.136191 (+ 1477us) server_negotiation.cc:407] Connection header received
0809 04:43:44.136406 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:44.136411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:44.136520 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:44.136572 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:44.137816 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:44.139056 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:44.141145 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:44.142912 (+ 1767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:44.143142 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:44.143228 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:44.143571 (+ 343us) server_negotiation.cc:299] Negotiation successful
0809 04:43:44.143716 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 04:43:44.144291 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:44.134748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:39399 (local address 127.12.234.1:43177)
0809 04:43:44.134912 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:44.134918 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:44.135575 (+ 657us) server_negotiation.cc:407] Connection header received
0809 04:43:44.136037 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:44.136043 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:44.136165 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:44.136278 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:43:44.137080 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:44.138580 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:44.141470 (+ 2890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:44.143019 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:44.143190 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:44.143264 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:44.143788 (+ 524us) server_negotiation.cc:299] Negotiation successful
0809 04:43:44.143948 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 04:43:44.144367 15743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:44.134222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:43177 (local address 127.12.234.2:39399)
0809 04:43:44.135443 (+ 1221us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:44.135465 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:44.135735 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:44.136317 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:44.136322 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:44.136369 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:44.136816 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:44.136832 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:44.138748 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:44.138753 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:44.141266 (+ 2513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:44.141283 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:44.143486 (+ 2203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:44.143490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:44.143617 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:44.143640 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 04:43:44.143755 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 04:43:44.143900 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":956,"mutex_wait_us":79,"thread_start_us":1385,"threads_started":1}
I20260809 04:43:44.145328 15262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d1c066df40e4ecd97562731c2ed0ef9" candidate_uuid: "32ac27385c334ce58e132403a2ef9711" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "646783dc4be145aab61bcb438398fa88" is_pre_election: true
I20260809 04:43:44.145627 15262 raft_consensus.cc:2454] T 2d1c066df40e4ecd97562731c2ed0ef9 P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32ac27385c334ce58e132403a2ef9711 in term 0.
I20260809 04:43:44.146045 15742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:44.134216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:40143 (local address 127.12.234.2:44893)
0809 04:43:44.136047 (+ 1831us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:44.136067 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:44.136225 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:44.136720 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:44.136726 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:44.136755 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:43:44.137599 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:44.137614 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:44.139198 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:44.139203 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:44.140997 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:44.141010 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:44.143134 (+ 2124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:44.143140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:44.143336 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:44.143374 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 04:43:44.144660 (+ 1286us) client_negotiation.cc:240] Negotiation successful
0809 04:43:44.144816 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1653,"mutex_wait_us":169,"thread_start_us":577,"threads_started":1}
I20260809 04:43:44.146425 15337 leader_election.cc:302] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32ac27385c334ce58e132403a2ef9711, 646783dc4be145aab61bcb438398fa88; no voters:
I20260809 04:43:44.146984 15722 raft_consensus.cc:2781] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:43:44.146909 15550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d1c066df40e4ecd97562731c2ed0ef9" candidate_uuid: "32ac27385c334ce58e132403a2ef9711" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "477443567ac448b4a2595113d452ef22" is_pre_election: true
I20260809 04:43:44.147126 15722 raft_consensus.cc:482] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:43:44.147264 15722 raft_consensus.cc:3037] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:44.147347 15550 raft_consensus.cc:2454] T 2d1c066df40e4ecd97562731c2ed0ef9 P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32ac27385c334ce58e132403a2ef9711 in term 0.
I20260809 04:43:44.190825 15722 raft_consensus.cc:504] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:44.192054 15722 leader_election.cc:288] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711 [CANDIDATE]: Term 1 election: Requested vote from peers 646783dc4be145aab61bcb438398fa88 (127.12.234.1:43177), 477443567ac448b4a2595113d452ef22 (127.12.234.3:40143)
I20260809 04:43:44.192895 15262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d1c066df40e4ecd97562731c2ed0ef9" candidate_uuid: "32ac27385c334ce58e132403a2ef9711" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "646783dc4be145aab61bcb438398fa88"
I20260809 04:43:44.192901 15550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d1c066df40e4ecd97562731c2ed0ef9" candidate_uuid: "32ac27385c334ce58e132403a2ef9711" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "477443567ac448b4a2595113d452ef22"
I20260809 04:43:44.193173 15262 raft_consensus.cc:3037] T 2d1c066df40e4ecd97562731c2ed0ef9 P 646783dc4be145aab61bcb438398fa88 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:44.193178 15550 raft_consensus.cc:3037] T 2d1c066df40e4ecd97562731c2ed0ef9 P 477443567ac448b4a2595113d452ef22 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:44.291625 15262 raft_consensus.cc:2454] T 2d1c066df40e4ecd97562731c2ed0ef9 P 646783dc4be145aab61bcb438398fa88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32ac27385c334ce58e132403a2ef9711 in term 1.
I20260809 04:43:44.292129 15550 raft_consensus.cc:2454] T 2d1c066df40e4ecd97562731c2ed0ef9 P 477443567ac448b4a2595113d452ef22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32ac27385c334ce58e132403a2ef9711 in term 1.
I20260809 04:43:44.293241 15335 leader_election.cc:302] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32ac27385c334ce58e132403a2ef9711, 477443567ac448b4a2595113d452ef22; no voters:
I20260809 04:43:44.293780 15722 raft_consensus.cc:2781] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:43:44.297773 15722 raft_consensus.cc:686] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711 [term 1 LEADER]: Becoming Leader. State: Replica: 32ac27385c334ce58e132403a2ef9711, State: Running, Role: LEADER
I20260809 04:43:44.298455 15722 consensus_queue.cc:237] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } }
I20260809 04:43:44.307369 15099 catalog_manager.cc:5495] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32ac27385c334ce58e132403a2ef9711 (127.12.234.2). New cstate: current_term: 1 leader_uuid: "32ac27385c334ce58e132403a2ef9711" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32ac27385c334ce58e132403a2ef9711" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 } health_report { overall_health: UNKNOWN } } }
I20260809 04:43:44.356196 15747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:44.347088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49430 (local address 127.12.234.2:46829)
0809 04:43:44.347926 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:44.347942 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:44.348672 (+ 730us) server_negotiation.cc:407] Connection header received
0809 04:43:44.348971 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:44.348975 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:44.349092 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:44.349310 (+ 218us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:43:44.350090 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:44.351112 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:44.353516 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:44.353993 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:44.354170 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:44.354673 (+ 503us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:43:44.355381 (+ 708us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:43:44.355559 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:44.355643 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 04:43:44.355735 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":430,"threads_started":1}
I20260809 04:43:44.377202 15751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:44.369656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:40143 (local address 127.12.234.2:47273)
0809 04:43:44.370468 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:44.370484 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:44.370618 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:44.371083 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:44.371087 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:44.371099 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:43:44.371548 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:44.371562 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:44.373193 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:44.373198 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:44.374609 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:44.374626 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:44.375828 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:44.375833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:44.375969 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:44.376056 (+ 87us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:43:44.376656 (+ 600us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:43:44.376667 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 04:43:44.376744 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 04:43:44.376871 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":472,"threads_started":1}
I20260809 04:43:44.377291 15732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:44.370018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:47273 (local address 127.12.234.3:40143)
0809 04:43:44.370251 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:44.370260 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:44.370591 (+ 331us) server_negotiation.cc:407] Connection header received
0809 04:43:44.370758 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:44.370768 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:44.370904 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:44.371038 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:43:44.371771 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:44.373004 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:44.374798 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:44.375568 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:44.375768 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:44.376333 (+ 565us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:43:44.376522 (+ 189us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:43:44.376640 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:44.376814 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 04:43:44.376936 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260809 04:43:44.387641 15550 raft_consensus.cc:1261] T 2d1c066df40e4ecd97562731c2ed0ef9 P 477443567ac448b4a2595113d452ef22 [term 1 FOLLOWER]: Refusing update from remote peer 32ac27385c334ce58e132403a2ef9711: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:43:44.387814 15262 raft_consensus.cc:1261] T 2d1c066df40e4ecd97562731c2ed0ef9 P 646783dc4be145aab61bcb438398fa88 [term 1 FOLLOWER]: Refusing update from remote peer 32ac27385c334ce58e132403a2ef9711: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:43:44.388619 15744 consensus_queue.cc:1035] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "477443567ac448b4a2595113d452ef22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:43:44.389045 15722 consensus_queue.cc:1035] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711 [LEADER]: Connected to new peer: Peer: permanent_uuid: "646783dc4be145aab61bcb438398fa88" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 43177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:43:44.428423 15747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:44.421213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:34867 (local address 127.12.234.2:46829)
0809 04:43:44.421471 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:44.421481 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:44.422068 (+ 587us) server_negotiation.cc:407] Connection header received
0809 04:43:44.422280 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:44.422285 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:44.422369 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:44.422476 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:43:44.423128 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:44.424262 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:44.426216 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:44.426821 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:44.426997 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:44.427399 (+ 402us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:43:44.427515 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:43:44.427614 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:44.427949 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 04:43:44.428043 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 04:43:44.428849 15755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:44.421146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:46829 (local address 127.12.234.3:34867)
0809 04:43:44.421958 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:44.421974 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:44.422142 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:44.422496 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:44.422500 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:44.422561 (+ 61us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:43:44.422956 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:44.422970 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:44.424438 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:44.424444 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:44.426060 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:44.426078 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:44.427012 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:44.427018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:44.427179 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:44.427225 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:43:44.427800 (+ 575us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:43:44.427812 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 04:43:44.427901 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 04:43:44.428051 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":491,"threads_started":1}
W20260809 04:43:44.430083 15682 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 04:43:46.070591 15100 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:43:46.071082 15100 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44440:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:43:46.077008 15100 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e2763f4b36424b518849a07d07a5f916: Sending DeleteTablet for 3 replicas of tablet 2d1c066df40e4ecd97562731c2ed0ef9
I20260809 04:43:46.078200 15239 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d1c066df40e4ecd97562731c2ed0ef9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:43:46 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:48502
I20260809 04:43:46.078267 15528 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d1c066df40e4ecd97562731c2ed0ef9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:43:46 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:45510
I20260809 04:43:46.078351 15383 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d1c066df40e4ecd97562731c2ed0ef9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:43:46 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:50914
I20260809 04:43:46.079183 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 15172
I20260809 04:43:46.089784 15780 tablet_replica.cc:330] T 2d1c066df40e4ecd97562731c2ed0ef9 P 646783dc4be145aab61bcb438398fa88: stopping tablet replica
I20260809 04:43:46.090200 15780 raft_consensus.cc:2229] T 2d1c066df40e4ecd97562731c2ed0ef9 P 646783dc4be145aab61bcb438398fa88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:43:46.091378 15782 tablet_replica.cc:330] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711: stopping tablet replica
I20260809 04:43:46.091777 15782 raft_consensus.cc:2229] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:43:46.092110 15781 tablet_replica.cc:330] T 2d1c066df40e4ecd97562731c2ed0ef9 P 477443567ac448b4a2595113d452ef22: stopping tablet replica
I20260809 04:43:46.092106 15780 raft_consensus.cc:2258] T 2d1c066df40e4ecd97562731c2ed0ef9 P 646783dc4be145aab61bcb438398fa88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:43:46.092501 15781 raft_consensus.cc:2229] T 2d1c066df40e4ecd97562731c2ed0ef9 P 477443567ac448b4a2595113d452ef22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:43:46.092625 15782 raft_consensus.cc:2258] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:43:46.093080 15781 raft_consensus.cc:2258] T 2d1c066df40e4ecd97562731c2ed0ef9 P 477443567ac448b4a2595113d452ef22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:43:46.093935 15780 ts_tablet_manager.cc:1905] T 2d1c066df40e4ecd97562731c2ed0ef9 P 646783dc4be145aab61bcb438398fa88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:43:46.094274 15782 ts_tablet_manager.cc:1905] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:43:46.094696 15781 ts_tablet_manager.cc:1905] T 2d1c066df40e4ecd97562731c2ed0ef9 P 477443567ac448b4a2595113d452ef22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:43:46.103282 15780 ts_tablet_manager.cc:1918] T 2d1c066df40e4ecd97562731c2ed0ef9 P 646783dc4be145aab61bcb438398fa88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 04:43:46.103596 15780 log.cc:1196] T 2d1c066df40e4ecd97562731c2ed0ef9 P 646783dc4be145aab61bcb438398fa88: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-0/wal/wals/2d1c066df40e4ecd97562731c2ed0ef9
I20260809 04:43:46.104027 15781 ts_tablet_manager.cc:1918] T 2d1c066df40e4ecd97562731c2ed0ef9 P 477443567ac448b4a2595113d452ef22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 04:43:46.104423 15781 log.cc:1196] T 2d1c066df40e4ecd97562731c2ed0ef9 P 477443567ac448b4a2595113d452ef22: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-2/wal/wals/2d1c066df40e4ecd97562731c2ed0ef9
I20260809 04:43:46.104609 15780 ts_tablet_manager.cc:1939] T 2d1c066df40e4ecd97562731c2ed0ef9 P 646783dc4be145aab61bcb438398fa88: Deleting consensus metadata
I20260809 04:43:46.104811 15782 ts_tablet_manager.cc:1918] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 04:43:46.105101 15782 log.cc:1196] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786250603326224-13224-0/minicluster-data/ts-1/wal/wals/2d1c066df40e4ecd97562731c2ed0ef9
I20260809 04:43:46.105168 15781 ts_tablet_manager.cc:1939] T 2d1c066df40e4ecd97562731c2ed0ef9 P 477443567ac448b4a2595113d452ef22: Deleting consensus metadata
I20260809 04:43:46.106035 15782 ts_tablet_manager.cc:1939] T 2d1c066df40e4ecd97562731c2ed0ef9 P 32ac27385c334ce58e132403a2ef9711: Deleting consensus metadata
I20260809 04:43:46.106178 15087 catalog_manager.cc:4841] TS 646783dc4be145aab61bcb438398fa88 (127.12.234.1:43177): tablet 2d1c066df40e4ecd97562731c2ed0ef9 (table test-table [id=52656bb6093546a2872613d7c18b58cb]) successfully deleted
I20260809 04:43:46.106447 15085 catalog_manager.cc:4841] TS 477443567ac448b4a2595113d452ef22 (127.12.234.3:40143): tablet 2d1c066df40e4ecd97562731c2ed0ef9 (table test-table [id=52656bb6093546a2872613d7c18b58cb]) successfully deleted
I20260809 04:43:46.108006 15084 catalog_manager.cc:4841] TS 32ac27385c334ce58e132403a2ef9711 (127.12.234.2:46829): tablet 2d1c066df40e4ecd97562731c2ed0ef9 (table test-table [id=52656bb6093546a2872613d7c18b58cb]) successfully deleted
Aug 09 04:43:46 dist-test-slave-1jjb krb5kdc[15054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250623, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.234.1@KRBTEST.COM
I20260809 04:43:46.128713 15784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:46.082847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45850 (local address 127.12.234.1:43177)
0809 04:43:46.087658 (+ 4811us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:46.087669 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:46.090505 (+ 2836us) server_negotiation.cc:407] Connection header received
0809 04:43:46.090614 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:46.090620 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:46.090748 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:46.090889 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:46.094076 (+ 3187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:46.096048 (+ 1972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:46.108527 (+ 12479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:46.109210 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:46.109384 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:46.121196 (+ 11812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:46.121215 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:46.121235 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:46.121279 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:46.125388 (+ 4109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:46.126217 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:46.126224 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:46.126227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:46.126481 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:46.126935 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:46.126938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:46.126940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:46.127434 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:46.127770 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:46.128039 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 04:43:46.128319 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4539,"thread_start_us":4227,"threads_started":1}
W20260809 04:43:46.129945 15306 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45850
W20260809 04:43:46.130843 13224 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 15172: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:43:46.131217 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 15172
I20260809 04:43:46.159632 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 15317
Aug 09 04:43:46 dist-test-slave-1jjb krb5kdc[15054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250623, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.234.2@KRBTEST.COM
I20260809 04:43:46.176750 15785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:46.160491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49442 (local address 127.12.234.2:46829)
0809 04:43:46.161109 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:46.161114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:46.161150 (+ 36us) server_negotiation.cc:407] Connection header received
0809 04:43:46.161207 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:46.161210 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:46.161274 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:46.161361 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:46.163574 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:46.164431 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:46.166011 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:46.166532 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:46.166631 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:46.171301 (+ 4670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:46.171317 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:46.171341 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:46.171378 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:46.174229 (+ 2851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:46.174845 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:46.174852 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:46.174854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:46.174992 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:46.175292 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:46.175295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:46.175297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:46.175657 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:46.175877 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:46.176283 (+ 406us) server_negotiation.cc:299] Negotiation successful
0809 04:43:46.176450 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":304,"threads_started":1}
W20260809 04:43:46.177413 15451 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49442
W20260809 04:43:46.178072 13224 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 15317: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:43:46.178252 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 15317
I20260809 04:43:46.207002 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 15461
Aug 09 04:43:46 dist-test-slave-1jjb krb5kdc[15054](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250623, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.234.3@KRBTEST.COM
I20260809 04:43:46.223628 15786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:46.207822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39186 (local address 127.12.234.3:40143)
0809 04:43:46.208371 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:46.208376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:46.208393 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:43:46.208449 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:46.208453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:46.208521 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:46.208621 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:46.209997 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:46.210735 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:46.212230 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:46.212625 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:46.212731 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:46.217734 (+ 5003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:46.217752 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:46.217773 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:46.217812 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:46.220983 (+ 3171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:46.221789 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:46.221797 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:46.221799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:46.221936 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:46.222296 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:46.222299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:46.222301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:46.222676 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:46.222922 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:46.223164 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 04:43:46.223346 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":313,"threads_started":1}
W20260809 04:43:46.224581 15598 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39186
W20260809 04:43:46.225346 13224 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 15461: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:43:46.225544 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 15461
I20260809 04:43:46.261103 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 15068
I20260809 04:43:46.262352 15141 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:43:46.386348 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 15068
2026-08-09T04:43:46Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9080 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:43:46 dist-test-slave-1jjb krb5kdc[15790](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:43:46 dist-test-slave-1jjb krb5kdc[15790](info): set up 2 sockets
Aug 09 04:43:46 dist-test-slave-1jjb krb5kdc[15790](info): commencing operation
krb5kdc: starting...
W20260809 04:43:47.574127 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.008s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:43:47 dist-test-slave-1jjb krb5kdc[15790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:43:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:43:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:43:47.799578 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:34943
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:43125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:34943
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:48.175659 15803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:48.176119 15803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:48.176393 15803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:48.188027 15803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:43:48.188392 15803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:48.188570 15803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:48.188777 15803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:43:48.188840 15803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:43:48.203595 15803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:43125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:34943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:34943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:48.205474 15803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:48.207216 15803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:48.225339 15810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:48.226140 15811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:48.226382 15813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:48.227960 15803 server_base.cc:1034] running on GCE node
I20260809 04:43:48.229081 15803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:48.230862 15803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:48.232182 15803 hybrid_clock.cc:648] HybridClock initialized: now 1786250628232040 us; error 46 us; skew 500 ppm
I20260809 04:43:48.232645 15803 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:48 dist-test-slave-1jjb krb5kdc[15790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250628, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:48.237720 15803 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:43:48.240018 15803 webserver.cc:460] Webserver started at http://127.12.234.62:46299/ using document root <none> and password file <none>
I20260809 04:43:48.240994 15803 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:48.241169 15803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:48.241547 15803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:48.245776 15803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "0e996d0448394a50938c75631634c91c"
format_stamp: "Formatted at 2026-08-09 04:43:48 on dist-test-slave-1jjb"
server_key: "abf79665074125c8866c41b6bd423054"
server_key_iv: "0f7faeb6177c81688cba6b09ca7e7c09"
server_key_version: "encryptionkey@0"
I20260809 04:43:48.246740 15803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "0e996d0448394a50938c75631634c91c"
format_stamp: "Formatted at 2026-08-09 04:43:48 on dist-test-slave-1jjb"
server_key: "abf79665074125c8866c41b6bd423054"
server_key_iv: "0f7faeb6177c81688cba6b09ca7e7c09"
server_key_version: "encryptionkey@0"
I20260809 04:43:48.254233 15803 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.000s sys 0.008s
I20260809 04:43:48.259117 15820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:48.260739 15803 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260809 04:43:48.260947 15803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "0e996d0448394a50938c75631634c91c"
format_stamp: "Formatted at 2026-08-09 04:43:48 on dist-test-slave-1jjb"
server_key: "abf79665074125c8866c41b6bd423054"
server_key_iv: "0f7faeb6177c81688cba6b09ca7e7c09"
server_key_version: "encryptionkey@0"
I20260809 04:43:48.261237 15803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:48.293531 15803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:48.300037 15803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:48.300516 15803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:48.352918 15803 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:34943
I20260809 04:43:48.352950 15875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:34943 every 8 connection(s)
I20260809 04:43:48.354713 15803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:43:48.355819 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 15803
I20260809 04:43:48.356297 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:43:48.357771 13224 external_mini_cluster.cc:1428] Setting key 81ddbc4f2d6b0fe2ac466b9c97681a7e
I20260809 04:43:48.362885 15876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 04:43:48 dist-test-slave-1jjb krb5kdc[15790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:48.379297 15876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e996d0448394a50938c75631634c91c: Bootstrap starting.
I20260809 04:43:48.383872 15876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e996d0448394a50938c75631634c91c: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:48.385169 15876 log.cc:824] T 00000000000000000000000000000000 P 0e996d0448394a50938c75631634c91c: Log is configured to *not* fsync() on all Append() calls
I20260809 04:43:48.388962 15876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e996d0448394a50938c75631634c91c: No bootstrap required, opened a new log
I20260809 04:43:48.389815 15879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:48.360669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33232 (local address 127.12.234.62:34943)
0809 04:43:48.362578 (+ 1909us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:48.362594 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:48.362674 (+ 80us) server_negotiation.cc:407] Connection header received
0809 04:43:48.366934 (+ 4260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:48.366966 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:48.367728 (+ 762us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:48.368364 (+ 636us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:48.369940 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:48.372711 (+ 2771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:48.374279 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:48.375362 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:48.375576 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:48.380199 (+ 4623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:48.380212 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:48.380233 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:48.380282 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:48.384503 (+ 4221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:48.385563 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:48.385572 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:48.385581 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:48.385838 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:48.386414 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:48.386422 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:48.386425 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:48.387194 (+ 769us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:48.387493 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:48.387950 (+ 457us) server_negotiation.cc:299] Negotiation successful
0809 04:43:48.388410 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1549,"thread_start_us":979,"threads_started":1}
I20260809 04:43:48.397511 15876 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e996d0448394a50938c75631634c91c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e996d0448394a50938c75631634c91c" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 34943 } }
I20260809 04:43:48.397889 15876 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e996d0448394a50938c75631634c91c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:48.397969 15876 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e996d0448394a50938c75631634c91c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e996d0448394a50938c75631634c91c, State: Initialized, Role: FOLLOWER
I20260809 04:43:48.398488 15876 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e996d0448394a50938c75631634c91c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e996d0448394a50938c75631634c91c" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 34943 } }
I20260809 04:43:48.398722 15876 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e996d0448394a50938c75631634c91c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:43:48.398867 15876 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e996d0448394a50938c75631634c91c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:43:48.399046 15876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e996d0448394a50938c75631634c91c [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:48.401731 15876 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e996d0448394a50938c75631634c91c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e996d0448394a50938c75631634c91c" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 34943 } }
I20260809 04:43:48.402278 15876 leader_election.cc:302] T 00000000000000000000000000000000 P 0e996d0448394a50938c75631634c91c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e996d0448394a50938c75631634c91c; no voters:
I20260809 04:43:48.402877 15876 leader_election.cc:288] T 00000000000000000000000000000000 P 0e996d0448394a50938c75631634c91c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:43:48.403031 15881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e996d0448394a50938c75631634c91c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:43:48.403780 15881 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e996d0448394a50938c75631634c91c [term 1 LEADER]: Becoming Leader. State: Replica: 0e996d0448394a50938c75631634c91c, State: Running, Role: LEADER
I20260809 04:43:48.404381 15881 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e996d0448394a50938c75631634c91c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e996d0448394a50938c75631634c91c" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 34943 } }
I20260809 04:43:48.405113 15876 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e996d0448394a50938c75631634c91c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:43:48.409047 15882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e996d0448394a50938c75631634c91c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e996d0448394a50938c75631634c91c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e996d0448394a50938c75631634c91c" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 34943 } } }
I20260809 04:43:48.409634 15882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e996d0448394a50938c75631634c91c [sys.catalog]: This master's current role is: LEADER
I20260809 04:43:48.409013 15884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e996d0448394a50938c75631634c91c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e996d0448394a50938c75631634c91c. Latest consensus state: current_term: 1 leader_uuid: "0e996d0448394a50938c75631634c91c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e996d0448394a50938c75631634c91c" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 34943 } } }
I20260809 04:43:48.410156 15884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e996d0448394a50938c75631634c91c [sys.catalog]: This master's current role is: LEADER
I20260809 04:43:48.411269 15892 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:43:48.416616 15892 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:43:48.425323 15892 catalog_manager.cc:1347] Generated new cluster ID: c7f6789d14124a47a7d2a7a46912a5a8
I20260809 04:43:48.425451 15892 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:43:48.440148 15892 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:43:48.442313 15892 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:43:48.451484 15892 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e996d0448394a50938c75631634c91c: Generated new TSK 0
I20260809 04:43:48.452689 15892 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
I20260809 04:43:48.516306 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:0
--local_ip_for_outbound_sockets=127.12.234.1
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:34943
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:43125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:48.868160 15904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:48.868458 15904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:48.868649 15904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:48.876314 15904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:48.876436 15904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:48.876655 15904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:43:48.887115 15904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:43125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:34943
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:48.888605 15904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:48.890012 15904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:48.904412 15909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:48.904487 15910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:48.904366 15912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:48.909927 15904 server_base.cc:1034] running on GCE node
I20260809 04:43:48.910799 15904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:48.911877 15904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:48.913084 15904 hybrid_clock.cc:648] HybridClock initialized: now 1786250628913028 us; error 78 us; skew 500 ppm
I20260809 04:43:48.913417 15904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:48 dist-test-slave-1jjb krb5kdc[15790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250628, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:48.916733 15904 init.cc:377] Logged in from keytab as kudu/127.12.234.1@KRBTEST.COM (short username kudu)
I20260809 04:43:48.918435 15904 webserver.cc:460] Webserver started at http://127.12.234.1:45103/ using document root <none> and password file <none>
I20260809 04:43:48.919209 15904 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:48.919341 15904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:48.919683 15904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:48.922895 15904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/data/instance:
uuid: "dff3d6bdcbe4419dbab7228a6899e60a"
format_stamp: "Formatted at 2026-08-09 04:43:48 on dist-test-slave-1jjb"
server_key: "66b8648a8500f3bb866e28fd6d551f2c"
server_key_iv: "21c756e4605a67a0550f92f58507b8b8"
server_key_version: "encryptionkey@0"
I20260809 04:43:48.923815 15904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance:
uuid: "dff3d6bdcbe4419dbab7228a6899e60a"
format_stamp: "Formatted at 2026-08-09 04:43:48 on dist-test-slave-1jjb"
server_key: "66b8648a8500f3bb866e28fd6d551f2c"
server_key_iv: "21c756e4605a67a0550f92f58507b8b8"
server_key_version: "encryptionkey@0"
I20260809 04:43:48.929646 15904 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 04:43:48.933255 15919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:48.934556 15904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 04:43:48.934728 15904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "dff3d6bdcbe4419dbab7228a6899e60a"
format_stamp: "Formatted at 2026-08-09 04:43:48 on dist-test-slave-1jjb"
server_key: "66b8648a8500f3bb866e28fd6d551f2c"
server_key_iv: "21c756e4605a67a0550f92f58507b8b8"
server_key_version: "encryptionkey@0"
I20260809 04:43:48.934931 15904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:48.948694 15904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:48.953553 15904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:48.953871 15904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:48.954867 15904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:43:48.956346 15904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:43:48.956446 15904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:48.956593 15904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:43:48.956678 15904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:48.991614 15904 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:34637
I20260809 04:43:48.991657 16037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:34637 every 8 connection(s)
I20260809 04:43:48.993031 15904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:43:49.000236 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 15904
I20260809 04:43:49.000610 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance
I20260809 04:43:49.001276 13224 external_mini_cluster.cc:1428] Setting key 4c924ea0af2ad991ac4402d7477f3506
Aug 09 04:43:49 dist-test-slave-1jjb krb5kdc[15790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250628, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:49.014318 16041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:48.995754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:51057 (local address 127.12.234.62:34943)
0809 04:43:48.996265 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:48.996269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:48.997104 (+ 835us) server_negotiation.cc:407] Connection header received
0809 04:43:48.998994 (+ 1890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:48.999000 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:48.999097 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:48.999217 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:49.000944 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:49.001747 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:49.003633 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:49.004055 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:49.004239 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:49.008254 (+ 4015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:49.008260 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:49.008263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:49.008297 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:49.011042 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:49.011825 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:49.011833 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:49.011841 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:49.011978 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:49.012481 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:49.012485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:49.012487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:49.012780 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:49.012941 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:49.013674 (+ 733us) server_negotiation.cc:299] Negotiation successful
0809 04:43:49.013907 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":272,"threads_started":1}
I20260809 04:43:49.015386 16042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:48.996218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:34943 (local address 127.12.234.1:51057)
0809 04:43:48.996929 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:48.996976 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:48.998712 (+ 1736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:48.999458 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:48.999466 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:49.000004 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:49.000757 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:49.000781 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:49.001915 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:49.001920 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:49.003489 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:49.003501 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:49.004342 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:49.004349 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:49.004590 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:49.005373 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:49.005397 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:49.007964 (+ 2567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:49.011195 (+ 3231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:49.011205 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:49.011225 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:49.011684 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:49.012118 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:49.012122 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:49.012126 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:49.012335 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:49.012915 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:49.012924 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:49.013304 (+ 380us) client_negotiation.cc:769] Sending connection context
0809 04:43:49.013578 (+ 274us) client_negotiation.cc:240] Negotiation successful
0809 04:43:49.014177 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":271,"threads_started":1}
I20260809 04:43:49.017135 16038 heartbeater.cc:347] Connected to a master server at 127.12.234.62:34943
I20260809 04:43:49.017488 16038 heartbeater.cc:464] Registering TS with master...
I20260809 04:43:49.018491 16038 heartbeater.cc:511] Master 127.12.234.62:34943 requested a full tablet report, sending...
I20260809 04:43:49.021216 15838 ts_manager.cc:194] Registered new tserver with Master: dff3d6bdcbe4419dbab7228a6899e60a (127.12.234.1:34637)
I20260809 04:43:49.023638 15838 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.1@KRBTEST.COM'} at 127.12.234.1:51057
WARNING: no policy specified for kudu/127.12.234.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
I20260809 04:43:49.057852 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:0
--local_ip_for_outbound_sockets=127.12.234.2
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:34943
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:43125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:49.402168 16045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:49.402472 16045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:49.402668 16045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:49.410868 16045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:49.411039 16045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:49.411180 16045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:43:49.421640 16045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:43125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:34943
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:49.422981 16045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:49.424311 16045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:49.438191 16050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:49.438194 16051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:49.438333 16053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:49.439082 16045 server_base.cc:1034] running on GCE node
I20260809 04:43:49.439738 16045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:49.440927 16045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:49.442190 16045 hybrid_clock.cc:648] HybridClock initialized: now 1786250629442147 us; error 66 us; skew 500 ppm
I20260809 04:43:49.442576 16045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:49 dist-test-slave-1jjb krb5kdc[15790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250629, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:49.445989 16045 init.cc:377] Logged in from keytab as kudu/127.12.234.2@KRBTEST.COM (short username kudu)
I20260809 04:43:49.447556 16045 webserver.cc:460] Webserver started at http://127.12.234.2:40041/ using document root <none> and password file <none>
I20260809 04:43:49.448365 16045 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:49.448513 16045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:49.448834 16045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:49.452040 16045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/data/instance:
uuid: "703474c6d6c146db8317c535d10bd523"
format_stamp: "Formatted at 2026-08-09 04:43:49 on dist-test-slave-1jjb"
server_key: "19e0d19830580ede86b5d83588ebfcad"
server_key_iv: "9013170b0d03f9c04ccfbb0d8691114d"
server_key_version: "encryptionkey@0"
I20260809 04:43:49.453044 16045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance:
uuid: "703474c6d6c146db8317c535d10bd523"
format_stamp: "Formatted at 2026-08-09 04:43:49 on dist-test-slave-1jjb"
server_key: "19e0d19830580ede86b5d83588ebfcad"
server_key_iv: "9013170b0d03f9c04ccfbb0d8691114d"
server_key_version: "encryptionkey@0"
I20260809 04:43:49.458678 16045 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 04:43:49.462519 16060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:49.463824 16045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 04:43:49.464030 16045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "703474c6d6c146db8317c535d10bd523"
format_stamp: "Formatted at 2026-08-09 04:43:49 on dist-test-slave-1jjb"
server_key: "19e0d19830580ede86b5d83588ebfcad"
server_key_iv: "9013170b0d03f9c04ccfbb0d8691114d"
server_key_version: "encryptionkey@0"
I20260809 04:43:49.464303 16045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:49.493063 16045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:49.497361 16045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:49.497685 16045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:49.498735 16045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:43:49.500164 16045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:43:49.500263 16045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:49.500417 16045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:43:49.500494 16045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:49.535240 16045 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:37981
I20260809 04:43:49.535327 16177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:37981 every 8 connection(s)
I20260809 04:43:49.536636 16045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:43:49.543164 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 16045
I20260809 04:43:49.543396 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance
I20260809 04:43:49.543860 13224 external_mini_cluster.cc:1428] Setting key 33cafbb21a7224f4ac9ff21fa2c1d687
Aug 09 04:43:49 dist-test-slave-1jjb krb5kdc[15790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250629, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:49.557736 16180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:49.539240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:41537 (local address 127.12.234.62:34943)
0809 04:43:49.539699 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:49.539704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:49.540601 (+ 897us) server_negotiation.cc:407] Connection header received
0809 04:43:49.542454 (+ 1853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:49.542460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:49.542551 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:49.542676 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:49.544406 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:49.545313 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:49.546988 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:49.547419 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:49.547551 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:49.551132 (+ 3581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:49.551137 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:49.551141 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:49.551174 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:49.554498 (+ 3324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:49.555240 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:49.555245 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:49.555249 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:49.555388 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:49.555841 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:49.555847 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:49.555850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:49.556200 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:49.556358 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:49.557100 (+ 742us) server_negotiation.cc:299] Negotiation successful
0809 04:43:49.557325 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":263,"threads_started":1}
I20260809 04:43:49.558880 16181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:49.539703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:34943 (local address 127.12.234.2:41537)
0809 04:43:49.540439 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:49.540488 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:49.542192 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:49.542929 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:49.542936 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:49.543570 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:49.544236 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:49.544256 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:49.545483 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:49.545487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:49.546832 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:49.546845 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:49.547659 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:49.547667 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:49.547891 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:49.548653 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:49.548681 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:49.550889 (+ 2208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:49.554660 (+ 3771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:49.554667 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:49.554681 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:49.555108 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:49.555524 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:49.555528 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:49.555531 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:49.555717 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:49.556332 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:49.556340 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:49.556700 (+ 360us) client_negotiation.cc:769] Sending connection context
0809 04:43:49.557005 (+ 305us) client_negotiation.cc:240] Negotiation successful
0809 04:43:49.557598 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":312,"threads_started":1}
I20260809 04:43:49.560657 16178 heartbeater.cc:347] Connected to a master server at 127.12.234.62:34943
I20260809 04:43:49.560985 16178 heartbeater.cc:464] Registering TS with master...
I20260809 04:43:49.561723 16178 heartbeater.cc:511] Master 127.12.234.62:34943 requested a full tablet report, sending...
I20260809 04:43:49.563333 15838 ts_manager.cc:194] Registered new tserver with Master: 703474c6d6c146db8317c535d10bd523 (127.12.234.2:37981)
I20260809 04:43:49.564828 15838 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.2@KRBTEST.COM'} at 127.12.234.2:41537
WARNING: no policy specified for kudu/127.12.234.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
I20260809 04:43:49.595690 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:0
--local_ip_for_outbound_sockets=127.12.234.3
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:34943
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:43125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:49.942718 16184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:49.942976 16184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:49.943159 16184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:49.950803 16184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:49.950964 16184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:49.951172 16184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:43:49.960964 16184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:43125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:34943
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:49.962245 16184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:49.963593 16184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:49.977300 16190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:49.977802 16192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:49.977833 16189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:49.978233 16184 server_base.cc:1034] running on GCE node
I20260809 04:43:49.978902 16184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:49.979872 16184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:49.981117 16184 hybrid_clock.cc:648] HybridClock initialized: now 1786250629981070 us; error 34 us; skew 500 ppm
I20260809 04:43:49.981465 16184 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:49 dist-test-slave-1jjb krb5kdc[15790](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250629, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:49.984732 16184 init.cc:377] Logged in from keytab as kudu/127.12.234.3@KRBTEST.COM (short username kudu)
I20260809 04:43:49.986126 16184 webserver.cc:460] Webserver started at http://127.12.234.3:40417/ using document root <none> and password file <none>
I20260809 04:43:49.986836 16184 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:49.986968 16184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:49.987317 16184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:49.990312 16184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/data/instance:
uuid: "02ff84494ff84d46a3f959cbbc2044c0"
format_stamp: "Formatted at 2026-08-09 04:43:49 on dist-test-slave-1jjb"
server_key: "10840485bc94fece4dbfcc95ea48f8b7"
server_key_iv: "2d01e8f165b532a7d87f94213054b889"
server_key_version: "encryptionkey@0"
I20260809 04:43:49.991174 16184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance:
uuid: "02ff84494ff84d46a3f959cbbc2044c0"
format_stamp: "Formatted at 2026-08-09 04:43:49 on dist-test-slave-1jjb"
server_key: "10840485bc94fece4dbfcc95ea48f8b7"
server_key_iv: "2d01e8f165b532a7d87f94213054b889"
server_key_version: "encryptionkey@0"
I20260809 04:43:49.996681 16184 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260809 04:43:50.000020 16199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:50.001307 16184 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 04:43:50.001487 16184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "02ff84494ff84d46a3f959cbbc2044c0"
format_stamp: "Formatted at 2026-08-09 04:43:49 on dist-test-slave-1jjb"
server_key: "10840485bc94fece4dbfcc95ea48f8b7"
server_key_iv: "2d01e8f165b532a7d87f94213054b889"
server_key_version: "encryptionkey@0"
I20260809 04:43:50.001695 16184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:50.023691 16184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:50.027302 16038 heartbeater.cc:503] Master 127.12.234.62:34943 was elected leader, sending a full tablet report...
I20260809 04:43:50.028378 16184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:50.028718 16184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:50.029735 16184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:43:50.030997 16184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:43:50.031075 16184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:50.031157 16184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:43:50.031201 16184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:50.064915 16184 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:42229
I20260809 04:43:50.064997 16319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:42229 every 8 connection(s)
I20260809 04:43:50.066234 16184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:43:50.076264 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 16184
I20260809 04:43:50.076462 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance
I20260809 04:43:50.076894 13224 external_mini_cluster.cc:1428] Setting key 3aae2eaf96bed4e46795e6bfc062d29d
Aug 09 04:43:50 dist-test-slave-1jjb krb5kdc[15790](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250629, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:50.085353 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:50.068731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:57561 (local address 127.12.234.62:34943)
0809 04:43:50.069198 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:50.069204 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:50.070322 (+ 1118us) server_negotiation.cc:407] Connection header received
0809 04:43:50.072211 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:50.072215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:50.072289 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:50.072381 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:50.073800 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:50.074573 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:50.076067 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:50.076569 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:50.076708 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:50.080293 (+ 3585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:50.080297 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:50.080301 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:50.080331 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:50.082769 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:50.083384 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:50.083388 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:50.083390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:50.083508 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:50.083864 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:50.083871 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:50.083875 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:50.084193 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:50.084312 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:50.084902 (+ 590us) server_negotiation.cc:299] Negotiation successful
0809 04:43:50.085071 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":226,"threads_started":1}
I20260809 04:43:50.086279 16323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:50.069352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:34943 (local address 127.12.234.3:57561)
0809 04:43:50.070135 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:50.070197 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:50.071907 (+ 1710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:50.072600 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:50.072607 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:50.073117 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:50.073656 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:50.073674 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:50.074700 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:50.074704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:50.075895 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:50.075907 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:50.076806 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:50.076813 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:50.077129 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:50.077830 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:50.077852 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:50.080015 (+ 2163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:50.082889 (+ 2874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:50.082895 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:50.082905 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:50.083287 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:50.083600 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:50.083603 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:50.083606 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:50.083772 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:50.084302 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:50.084308 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:50.084602 (+ 294us) client_negotiation.cc:769] Sending connection context
0809 04:43:50.084824 (+ 222us) client_negotiation.cc:240] Negotiation successful
0809 04:43:50.085184 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":265,"threads_started":1}
I20260809 04:43:50.087574 16320 heartbeater.cc:347] Connected to a master server at 127.12.234.62:34943
I20260809 04:43:50.087867 16320 heartbeater.cc:464] Registering TS with master...
I20260809 04:43:50.088624 16320 heartbeater.cc:511] Master 127.12.234.62:34943 requested a full tablet report, sending...
I20260809 04:43:50.090003 15838 ts_manager.cc:194] Registered new tserver with Master: 02ff84494ff84d46a3f959cbbc2044c0 (127.12.234.3:42229)
I20260809 04:43:50.090968 15838 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.3@KRBTEST.COM'} at 127.12.234.3:57561
I20260809 04:43:50.091871 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 04:43:50.127800 16322 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:43:50.125317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33240 (local address 127.12.234.62:34943)
0809 04:43:50.125582 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:50.125592 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:50.125920 (+ 328us) server_negotiation.cc:407] Connection header received
0809 04:43:50.126142 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:50.126148 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:50.126211 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:50.126323 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:50.127595 (+ 1272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33240: BlockingRecv error: recv got EOF from 127.0.0.1:33240 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:43:50.131246 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 15904
W20260809 04:43:50.133814 13224 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 15904: Not authorized: Client connection negotiation failed: client connection to 127.12.234.1:34637: server requires authentication, but client does not have Kerberos credentials available
I20260809 04:43:50.134014 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 15904
I20260809 04:43:50.152529 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 16045
W20260809 04:43:50.155014 13224 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 16045: Not authorized: Client connection negotiation failed: client connection to 127.12.234.2:37981: server requires authentication, but client does not have Kerberos credentials available
I20260809 04:43:50.155187 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 16045
I20260809 04:43:50.172339 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 16184
W20260809 04:43:50.175153 13224 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 16184: Not authorized: Client connection negotiation failed: client connection to 127.12.234.3:42229: server requires authentication, but client does not have Kerberos credentials available
I20260809 04:43:50.175293 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 16184
I20260809 04:43:50.194523 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 15803
I20260809 04:43:50.195889 15870 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:43:50.298532 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 15803
2026-08-09T04:43:50Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3908 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:43:50 dist-test-slave-1jjb krb5kdc[16339](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:43:50 dist-test-slave-1jjb krb5kdc[16339](info): set up 2 sockets
Aug 09 04:43:50 dist-test-slave-1jjb krb5kdc[16339](info): commencing operation
krb5kdc: starting...
W20260809 04:43:51.485726 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.007s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:43:51 dist-test-slave-1jjb krb5kdc[16339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:43:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:43:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:43:51.688567 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:43185
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:39397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:43185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:52.025096 16352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:52.025527 16352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:52.026057 16352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:52.037333 16352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:43:52.037601 16352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:52.037750 16352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:52.037884 16352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:43:52.037990 16352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:43:52.050271 16352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:39397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:43185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:43185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:52.051816 16352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:52.053303 16352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:52.067775 16357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:52.067670 16358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:52.068020 16360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:52.069101 16352 server_base.cc:1034] running on GCE node
I20260809 04:43:52.070309 16352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:52.072263 16352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:52.073565 16352 hybrid_clock.cc:648] HybridClock initialized: now 1786250632073471 us; error 92 us; skew 500 ppm
I20260809 04:43:52.074059 16352 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:52 dist-test-slave-1jjb krb5kdc[16339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250632, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:52.082726 16352 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:43:52.084666 16352 webserver.cc:460] Webserver started at http://127.12.234.62:42525/ using document root <none> and password file <none>
I20260809 04:43:52.085420 16352 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:52.085532 16352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:52.085824 16352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:52.089308 16352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "2df3442d1d3841f9ad32a7cc7043d20d"
format_stamp: "Formatted at 2026-08-09 04:43:52 on dist-test-slave-1jjb"
server_key: "e146650deceb7d07efab4583fd66462d"
server_key_iv: "3611bcba5b63a55896ff17cc9138c7ce"
server_key_version: "encryptionkey@0"
I20260809 04:43:52.090138 16352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "2df3442d1d3841f9ad32a7cc7043d20d"
format_stamp: "Formatted at 2026-08-09 04:43:52 on dist-test-slave-1jjb"
server_key: "e146650deceb7d07efab4583fd66462d"
server_key_iv: "3611bcba5b63a55896ff17cc9138c7ce"
server_key_version: "encryptionkey@0"
I20260809 04:43:52.096446 16352 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260809 04:43:52.100517 16368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:52.102161 16352 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:43:52.102339 16352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "2df3442d1d3841f9ad32a7cc7043d20d"
format_stamp: "Formatted at 2026-08-09 04:43:52 on dist-test-slave-1jjb"
server_key: "e146650deceb7d07efab4583fd66462d"
server_key_iv: "3611bcba5b63a55896ff17cc9138c7ce"
server_key_version: "encryptionkey@0"
I20260809 04:43:52.102721 16352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:52.142328 16352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:52.147661 16352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:52.148016 16352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:52.170068 16352 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:43185
I20260809 04:43:52.170087 16421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:43185 every 8 connection(s)
I20260809 04:43:52.171603 16352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:43:52.172652 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 16352
I20260809 04:43:52.172926 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:43:52.173583 13224 external_mini_cluster.cc:1428] Setting key cb6c4f27c6c1572dc5816fa9d74c6c07
I20260809 04:43:52.176460 16422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 04:43:52 dist-test-slave-1jjb krb5kdc[16339](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:52.195745 16422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2df3442d1d3841f9ad32a7cc7043d20d: Bootstrap starting.
I20260809 04:43:52.199795 16422 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2df3442d1d3841f9ad32a7cc7043d20d: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:52.201149 16422 log.cc:824] T 00000000000000000000000000000000 P 2df3442d1d3841f9ad32a7cc7043d20d: Log is configured to *not* fsync() on all Append() calls
I20260809 04:43:52.205435 16422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2df3442d1d3841f9ad32a7cc7043d20d: No bootstrap required, opened a new log
I20260809 04:43:52.205904 16425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:52.175567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38628 (local address 127.12.234.62:43185)
0809 04:43:52.176655 (+ 1088us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:52.176668 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:52.176710 (+ 42us) server_negotiation.cc:407] Connection header received
0809 04:43:52.178400 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:52.178421 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:52.178792 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:52.179151 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:52.181407 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:52.183879 (+ 2472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:52.186555 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:52.187606 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:52.187859 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:52.196667 (+ 8808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:52.196680 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:52.196709 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:52.196771 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:52.200546 (+ 3775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:52.201492 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:52.201503 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:52.201512 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:52.201781 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:52.202326 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:52.202330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:52.202332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:52.203059 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:52.203384 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:52.203914 (+ 530us) server_negotiation.cc:299] Negotiation successful
0809 04:43:52.204431 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":801,"thread_start_us":525,"threads_started":1}
I20260809 04:43:52.211870 13224 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:43:52.218093 16422 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2df3442d1d3841f9ad32a7cc7043d20d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2df3442d1d3841f9ad32a7cc7043d20d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 43185 } }
I20260809 04:43:52.218748 16422 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2df3442d1d3841f9ad32a7cc7043d20d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:52.218875 16422 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2df3442d1d3841f9ad32a7cc7043d20d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2df3442d1d3841f9ad32a7cc7043d20d, State: Initialized, Role: FOLLOWER
I20260809 04:43:52.219695 16422 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2df3442d1d3841f9ad32a7cc7043d20d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2df3442d1d3841f9ad32a7cc7043d20d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 43185 } }
I20260809 04:43:52.220124 16422 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2df3442d1d3841f9ad32a7cc7043d20d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:43:52.220307 16422 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2df3442d1d3841f9ad32a7cc7043d20d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:43:52.220516 16422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2df3442d1d3841f9ad32a7cc7043d20d [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:52.223923 16422 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2df3442d1d3841f9ad32a7cc7043d20d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2df3442d1d3841f9ad32a7cc7043d20d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 43185 } }
I20260809 04:43:52.224581 16422 leader_election.cc:302] T 00000000000000000000000000000000 P 2df3442d1d3841f9ad32a7cc7043d20d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2df3442d1d3841f9ad32a7cc7043d20d; no voters:
I20260809 04:43:52.225389 16422 leader_election.cc:288] T 00000000000000000000000000000000 P 2df3442d1d3841f9ad32a7cc7043d20d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:43:52.225525 16428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2df3442d1d3841f9ad32a7cc7043d20d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:43:52.226893 16428 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2df3442d1d3841f9ad32a7cc7043d20d [term 1 LEADER]: Becoming Leader. State: Replica: 2df3442d1d3841f9ad32a7cc7043d20d, State: Running, Role: LEADER
I20260809 04:43:52.227608 16428 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2df3442d1d3841f9ad32a7cc7043d20d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2df3442d1d3841f9ad32a7cc7043d20d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 43185 } }
I20260809 04:43:52.228148 16422 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2df3442d1d3841f9ad32a7cc7043d20d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:43:52.231036 16430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2df3442d1d3841f9ad32a7cc7043d20d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2df3442d1d3841f9ad32a7cc7043d20d. Latest consensus state: current_term: 1 leader_uuid: "2df3442d1d3841f9ad32a7cc7043d20d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2df3442d1d3841f9ad32a7cc7043d20d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 43185 } } }
I20260809 04:43:52.231526 16430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2df3442d1d3841f9ad32a7cc7043d20d [sys.catalog]: This master's current role is: LEADER
I20260809 04:43:52.232299 16429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2df3442d1d3841f9ad32a7cc7043d20d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2df3442d1d3841f9ad32a7cc7043d20d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2df3442d1d3841f9ad32a7cc7043d20d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 43185 } } }
I20260809 04:43:52.232764 16429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2df3442d1d3841f9ad32a7cc7043d20d [sys.catalog]: This master's current role is: LEADER
I20260809 04:43:52.236411 16435 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:43:52.242903 16435 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:43:52.269608 16435 catalog_manager.cc:1347] Generated new cluster ID: 347e28c7733c4319ab6bfe470d45cddd
I20260809 04:43:52.269802 16435 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:43:52.282075 16435 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:43:52.283886 16435 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:43:52.286441 16425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:52.275901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38630 (local address 127.12.234.62:43185)
0809 04:43:52.276134 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:52.276142 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:52.276327 (+ 185us) server_negotiation.cc:407] Connection header received
0809 04:43:52.276723 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:52.276730 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:52.276848 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:52.276993 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:52.278652 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:52.279767 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:52.281990 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:52.282704 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:52.282878 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:52.284267 (+ 1389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:52.284275 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:52.284284 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:43:52.285383 (+ 1099us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:52.285472 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:43:52.285512 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:52.285632 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:52.285824 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 04:43:52.285973 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":62}
I20260809 04:43:52.300541 16435 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2df3442d1d3841f9ad32a7cc7043d20d: Generated new TSK 0
I20260809 04:43:52.301638 16435 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:43:52.315986 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 16352
I20260809 04:43:52.317962 16417 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:43:52.415712 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 16352
2026-08-09T04:43:52Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2112 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:43:52 dist-test-slave-1jjb krb5kdc[16457](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:43:52 dist-test-slave-1jjb krb5kdc[16457](info): set up 2 sockets
Aug 09 04:43:52 dist-test-slave-1jjb krb5kdc[16457](info): commencing operation
krb5kdc: starting...
W20260809 04:43:53.611828 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:43:53 dist-test-slave-1jjb krb5kdc[16457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:43:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:43:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:43:53.893388 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:36487
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:35669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:36487
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:54.255573 16471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:54.255863 16471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:54.255986 16471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:54.263031 16471 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:43:54.263160 16471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:54.263208 16471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:54.263336 16471 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:43:54.263437 16471 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:43:54.273424 16471 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:35669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:36487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:36487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:54.274852 16471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:54.276264 16471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:54.287643 16476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:54.287498 16477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:54.287724 16479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:54.288825 16471 server_base.cc:1034] running on GCE node
I20260809 04:43:54.289577 16471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:54.291019 16471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:54.292261 16471 hybrid_clock.cc:648] HybridClock initialized: now 1786250634292226 us; error 49 us; skew 500 ppm
I20260809 04:43:54.292598 16471 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:54 dist-test-slave-1jjb krb5kdc[16457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250634, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:54.295614 16471 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:43:54.297003 16471 webserver.cc:460] Webserver started at http://127.12.234.62:44467/ using document root <none> and password file <none>
I20260809 04:43:54.297689 16471 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:54.297835 16471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:54.298147 16471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:54.301047 16471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "5773f09c659d4a808346534a310c6581"
format_stamp: "Formatted at 2026-08-09 04:43:54 on dist-test-slave-1jjb"
server_key: "aac38a1fa49cf8d9fbe007d4b8dd1541"
server_key_iv: "bd59572f858dab8e48e247735ab6ecfb"
server_key_version: "encryptionkey@0"
I20260809 04:43:54.301811 16471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "5773f09c659d4a808346534a310c6581"
format_stamp: "Formatted at 2026-08-09 04:43:54 on dist-test-slave-1jjb"
server_key: "aac38a1fa49cf8d9fbe007d4b8dd1541"
server_key_iv: "bd59572f858dab8e48e247735ab6ecfb"
server_key_version: "encryptionkey@0"
I20260809 04:43:54.307065 16471 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 04:43:54.310870 16488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:54.312505 16471 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:43:54.312716 16471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "5773f09c659d4a808346534a310c6581"
format_stamp: "Formatted at 2026-08-09 04:43:54 on dist-test-slave-1jjb"
server_key: "aac38a1fa49cf8d9fbe007d4b8dd1541"
server_key_iv: "bd59572f858dab8e48e247735ab6ecfb"
server_key_version: "encryptionkey@0"
I20260809 04:43:54.312937 16471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:54.359062 16471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:54.363315 16471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:54.363581 16471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:54.382671 16471 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:36487
I20260809 04:43:54.382704 16539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:36487 every 8 connection(s)
I20260809 04:43:54.384128 16471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:43:54.388027 16540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:54.389887 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 16471
I20260809 04:43:54.390126 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:43:54.390785 13224 external_mini_cluster.cc:1428] Setting key 80e9a0358eb6d2f3d1ca2dfe92f73f6b
I20260809 04:43:54.401693 16540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5773f09c659d4a808346534a310c6581: Bootstrap starting.
I20260809 04:43:54.405200 16540 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5773f09c659d4a808346534a310c6581: Neither blocks nor log segments found. Creating new log.
Aug 09 04:43:54 dist-test-slave-1jjb krb5kdc[16457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:54.406495 16540 log.cc:824] T 00000000000000000000000000000000 P 5773f09c659d4a808346534a310c6581: Log is configured to *not* fsync() on all Append() calls
I20260809 04:43:54.410094 16540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5773f09c659d4a808346534a310c6581: No bootstrap required, opened a new log
I20260809 04:43:54.415280 16543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:54.393006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58664 (local address 127.12.234.62:36487)
0809 04:43:54.393871 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:54.393887 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:54.393944 (+ 57us) server_negotiation.cc:407] Connection header received
0809 04:43:54.396323 (+ 2379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:54.396350 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:54.396845 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:54.397312 (+ 467us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:54.398553 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:54.400649 (+ 2096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:54.401916 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:54.402718 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:54.402912 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:54.407586 (+ 4674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:54.407602 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:54.407626 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:54.407677 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:54.410985 (+ 3308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:54.411859 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:54.411865 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:54.411871 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:54.412047 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:54.412501 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:54.412504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:54.412506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:54.413098 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:54.413336 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:54.413793 (+ 457us) server_negotiation.cc:299] Negotiation successful
0809 04:43:54.414180 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":391,"threads_started":1}
I20260809 04:43:54.419706 16540 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5773f09c659d4a808346534a310c6581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5773f09c659d4a808346534a310c6581" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36487 } }
I20260809 04:43:54.420157 16540 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5773f09c659d4a808346534a310c6581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:54.420276 16540 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5773f09c659d4a808346534a310c6581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5773f09c659d4a808346534a310c6581, State: Initialized, Role: FOLLOWER
I20260809 04:43:54.420840 16540 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5773f09c659d4a808346534a310c6581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5773f09c659d4a808346534a310c6581" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36487 } }
I20260809 04:43:54.421090 16540 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5773f09c659d4a808346534a310c6581 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:43:54.421275 16540 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5773f09c659d4a808346534a310c6581 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:43:54.421458 16540 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5773f09c659d4a808346534a310c6581 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:54.423743 16540 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5773f09c659d4a808346534a310c6581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5773f09c659d4a808346534a310c6581" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36487 } }
I20260809 04:43:54.424376 16540 leader_election.cc:302] T 00000000000000000000000000000000 P 5773f09c659d4a808346534a310c6581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5773f09c659d4a808346534a310c6581; no voters:
I20260809 04:43:54.425030 16540 leader_election.cc:288] T 00000000000000000000000000000000 P 5773f09c659d4a808346534a310c6581 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:43:54.425170 16545 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5773f09c659d4a808346534a310c6581 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:43:54.425881 16545 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5773f09c659d4a808346534a310c6581 [term 1 LEADER]: Becoming Leader. State: Replica: 5773f09c659d4a808346534a310c6581, State: Running, Role: LEADER
I20260809 04:43:54.426467 16545 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5773f09c659d4a808346534a310c6581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5773f09c659d4a808346534a310c6581" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36487 } }
I20260809 04:43:54.427300 16540 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5773f09c659d4a808346534a310c6581 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:43:54.429513 16547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5773f09c659d4a808346534a310c6581 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5773f09c659d4a808346534a310c6581. Latest consensus state: current_term: 1 leader_uuid: "5773f09c659d4a808346534a310c6581" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5773f09c659d4a808346534a310c6581" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36487 } } }
I20260809 04:43:54.429975 16547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5773f09c659d4a808346534a310c6581 [sys.catalog]: This master's current role is: LEADER
I20260809 04:43:54.429894 16546 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5773f09c659d4a808346534a310c6581 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5773f09c659d4a808346534a310c6581" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5773f09c659d4a808346534a310c6581" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36487 } } }
I20260809 04:43:54.430243 16546 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5773f09c659d4a808346534a310c6581 [sys.catalog]: This master's current role is: LEADER
I20260809 04:43:54.431212 16553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:43:54.436810 16553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:43:54.446179 16553 catalog_manager.cc:1347] Generated new cluster ID: b237cbc75cfa4e008c5fe7ff8a43b0d2
I20260809 04:43:54.446342 16553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:43:54.470237 16553 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:43:54.471671 16553 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:43:54.479378 16553 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5773f09c659d4a808346534a310c6581: Generated new TSK 0
I20260809 04:43:54.480333 16553 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
I20260809 04:43:54.560930 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:0
--local_ip_for_outbound_sockets=127.12.234.1
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:36487
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:35669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:54.914896 16568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:54.915210 16568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:54.915378 16568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:54.923168 16568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:54.923297 16568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:54.923493 16568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:43:54.934347 16568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:35669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:36487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:54.935940 16568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:54.937536 16568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:54.955360 16573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:54.955403 16574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:54.955307 16576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:54.956691 16568 server_base.cc:1034] running on GCE node
I20260809 04:43:54.957547 16568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:54.958748 16568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:54.960036 16568 hybrid_clock.cc:648] HybridClock initialized: now 1786250634959990 us; error 51 us; skew 500 ppm
I20260809 04:43:54.960547 16568 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:54 dist-test-slave-1jjb krb5kdc[16457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250634, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:54.964254 16568 init.cc:377] Logged in from keytab as kudu/127.12.234.1@KRBTEST.COM (short username kudu)
I20260809 04:43:54.966249 16568 webserver.cc:460] Webserver started at http://127.12.234.1:46791/ using document root <none> and password file <none>
I20260809 04:43:54.967047 16568 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:54.967204 16568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:54.967531 16568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:54.971933 16568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/data/instance:
uuid: "41f2cc4c50c441c4a610560cea13d703"
format_stamp: "Formatted at 2026-08-09 04:43:54 on dist-test-slave-1jjb"
server_key: "9aae8482a3727086f1c57bda9e093358"
server_key_iv: "c1508061b5fbdef13660c7cfd5a6d17f"
server_key_version: "encryptionkey@0"
I20260809 04:43:54.973443 16568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance:
uuid: "41f2cc4c50c441c4a610560cea13d703"
format_stamp: "Formatted at 2026-08-09 04:43:54 on dist-test-slave-1jjb"
server_key: "9aae8482a3727086f1c57bda9e093358"
server_key_iv: "c1508061b5fbdef13660c7cfd5a6d17f"
server_key_version: "encryptionkey@0"
I20260809 04:43:54.983461 16568 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.005s sys 0.004s
I20260809 04:43:54.989455 16584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:54.991106 16568 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:43:54.991325 16568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "41f2cc4c50c441c4a610560cea13d703"
format_stamp: "Formatted at 2026-08-09 04:43:54 on dist-test-slave-1jjb"
server_key: "9aae8482a3727086f1c57bda9e093358"
server_key_iv: "c1508061b5fbdef13660c7cfd5a6d17f"
server_key_version: "encryptionkey@0"
I20260809 04:43:54.991621 16568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:55.029793 16568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:55.035360 16568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:55.035722 16568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:55.037299 16568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:43:55.038851 16568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:43:55.038962 16568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:55.039129 16568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:43:55.039203 16568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:55.079377 16568 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:42875
I20260809 04:43:55.079474 16699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:42875 every 8 connection(s)
I20260809 04:43:55.080865 16568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:43:55.081522 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 16568
I20260809 04:43:55.081753 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance
I20260809 04:43:55.082634 13224 external_mini_cluster.cc:1428] Setting key b084aea889585aacdbef51f0b4231972
Aug 09 04:43:55 dist-test-slave-1jjb krb5kdc[16457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250634, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:55.107518 16702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:55.083994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:59641 (local address 127.12.234.62:36487)
0809 04:43:55.084506 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:55.084512 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:55.085731 (+ 1219us) server_negotiation.cc:407] Connection header received
0809 04:43:55.087761 (+ 2030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:55.087769 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:55.087869 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:55.088000 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:55.089905 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:55.091145 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:55.093753 (+ 2608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:55.094247 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:55.094388 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:55.100726 (+ 6338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:55.100733 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:55.100737 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:55.100783 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:55.103901 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:55.104909 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:55.104915 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:55.104917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:55.105065 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:55.105565 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:55.105569 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:55.105571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:55.105859 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:55.106029 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:55.106853 (+ 824us) server_negotiation.cc:299] Negotiation successful
0809 04:43:55.107115 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":276,"threads_started":1}
I20260809 04:43:55.108767 16703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:55.084668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:36487 (local address 127.12.234.1:59641)
0809 04:43:55.085537 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:55.085585 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:55.087415 (+ 1830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:55.088306 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:55.088316 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:55.089032 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:55.089695 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:55.089715 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:55.091506 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:55.091513 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:55.093450 (+ 1937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:55.093482 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:55.094581 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:55.094591 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:55.095208 (+ 617us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:55.096447 (+ 1239us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:55.096492 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:55.100368 (+ 3876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:55.104136 (+ 3768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:55.104148 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:55.104171 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:55.104735 (+ 564us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:55.105192 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:55.105197 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:55.105201 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:55.105415 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:55.106008 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:55.106017 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:55.106440 (+ 423us) client_negotiation.cc:769] Sending connection context
0809 04:43:55.106744 (+ 304us) client_negotiation.cc:240] Negotiation successful
0809 04:43:55.107405 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":376,"threads_started":1}
WARNING: no policy specified for kudu/127.12.234.2@KRBTEST.COM; defaulting to no policy
I20260809 04:43:55.110947 16700 heartbeater.cc:347] Connected to a master server at 127.12.234.62:36487
I20260809 04:43:55.111403 16700 heartbeater.cc:464] Registering TS with master...
I20260809 04:43:55.112375 16700 heartbeater.cc:511] Master 127.12.234.62:36487 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.2@KRBTEST.COM" created.
I20260809 04:43:55.115863 16505 ts_manager.cc:194] Registered new tserver with Master: 41f2cc4c50c441c4a610560cea13d703 (127.12.234.1:42875)
I20260809 04:43:55.117748 16505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.1@KRBTEST.COM'} at 127.12.234.1:59641
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
I20260809 04:43:55.147715 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:0
--local_ip_for_outbound_sockets=127.12.234.2
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:36487
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:35669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:55.535152 16706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:55.535583 16706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:55.535775 16706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:55.545912 16706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:55.546088 16706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:55.546307 16706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:43:55.557312 16706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:35669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:36487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:55.558776 16706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:55.560220 16706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:55.574740 16711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:55.574790 16712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:55.574831 16714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:55.575769 16706 server_base.cc:1034] running on GCE node
I20260809 04:43:55.576509 16706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:55.577564 16706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:55.578779 16706 hybrid_clock.cc:648] HybridClock initialized: now 1786250635578728 us; error 40 us; skew 500 ppm
I20260809 04:43:55.579133 16706 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:55 dist-test-slave-1jjb krb5kdc[16457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250635, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:55.582664 16706 init.cc:377] Logged in from keytab as kudu/127.12.234.2@KRBTEST.COM (short username kudu)
I20260809 04:43:55.584486 16706 webserver.cc:460] Webserver started at http://127.12.234.2:40571/ using document root <none> and password file <none>
I20260809 04:43:55.585219 16706 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:55.585325 16706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:55.585701 16706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:55.589274 16706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/data/instance:
uuid: "bcfece602314479fa748b61365165f4d"
format_stamp: "Formatted at 2026-08-09 04:43:55 on dist-test-slave-1jjb"
server_key: "e5b2272ebee0b8b77166d06b7a40f31d"
server_key_iv: "b2f872eb68c6e63c9e011829d9aa01fd"
server_key_version: "encryptionkey@0"
I20260809 04:43:55.590188 16706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance:
uuid: "bcfece602314479fa748b61365165f4d"
format_stamp: "Formatted at 2026-08-09 04:43:55 on dist-test-slave-1jjb"
server_key: "e5b2272ebee0b8b77166d06b7a40f31d"
server_key_iv: "b2f872eb68c6e63c9e011829d9aa01fd"
server_key_version: "encryptionkey@0"
I20260809 04:43:55.596068 16706 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 04:43:55.600163 16722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:55.601615 16706 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:43:55.601807 16706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "bcfece602314479fa748b61365165f4d"
format_stamp: "Formatted at 2026-08-09 04:43:55 on dist-test-slave-1jjb"
server_key: "e5b2272ebee0b8b77166d06b7a40f31d"
server_key_iv: "b2f872eb68c6e63c9e011829d9aa01fd"
server_key_version: "encryptionkey@0"
I20260809 04:43:55.602023 16706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:55.616194 16706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:55.620568 16706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:55.620870 16706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:55.621850 16706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:43:55.623227 16706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:43:55.623330 16706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:55.623476 16706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:43:55.623559 16706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:55.659353 16706 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:43213
I20260809 04:43:55.659401 16839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:43213 every 8 connection(s)
I20260809 04:43:55.660797 16706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:43:55.665540 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 16706
I20260809 04:43:55.665733 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance
I20260809 04:43:55.666337 13224 external_mini_cluster.cc:1428] Setting key cf980d0494ca929d5b4cfa41506ad937
Aug 09 04:43:55 dist-test-slave-1jjb krb5kdc[16457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250635, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:55.685928 16842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:55.663451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:37615 (local address 127.12.234.62:36487)
0809 04:43:55.663933 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:55.663939 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:55.664926 (+ 987us) server_negotiation.cc:407] Connection header received
0809 04:43:55.667480 (+ 2554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:55.667496 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:55.667748 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:55.667943 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:55.670154 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:55.670936 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:55.673513 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:55.673943 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:55.674059 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:55.678501 (+ 4442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:55.678509 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:55.678517 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:55.678589 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:55.682324 (+ 3735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:55.683087 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:55.683095 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:55.683102 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:55.683256 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:55.683732 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:55.683740 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:55.683746 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:55.684195 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:55.684343 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:55.685137 (+ 794us) server_negotiation.cc:299] Negotiation successful
0809 04:43:55.685427 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":229,"threads_started":1}
I20260809 04:43:55.686683 16843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:55.663964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:36487 (local address 127.12.234.2:37615)
0809 04:43:55.664762 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:55.664809 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:55.667087 (+ 2278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:55.668269 (+ 1182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:55.668281 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:55.668960 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:55.669898 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:55.669928 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:55.671114 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:55.671122 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:55.673345 (+ 2223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:55.673363 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:55.674184 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:55.674194 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:55.674553 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:55.675345 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:55.675379 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:55.678221 (+ 2842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:55.682489 (+ 4268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:55.682497 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:55.682512 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:55.682935 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:55.683369 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:55.683373 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:55.683375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:55.683591 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:55.684339 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:55.684345 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:55.684696 (+ 351us) client_negotiation.cc:769] Sending connection context
0809 04:43:55.684960 (+ 264us) client_negotiation.cc:240] Negotiation successful
0809 04:43:55.685536 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":278,"threads_started":1}
I20260809 04:43:55.688398 16840 heartbeater.cc:347] Connected to a master server at 127.12.234.62:36487
I20260809 04:43:55.688748 16840 heartbeater.cc:464] Registering TS with master...
I20260809 04:43:55.689558 16840 heartbeater.cc:511] Master 127.12.234.62:36487 requested a full tablet report, sending...
I20260809 04:43:55.691223 16505 ts_manager.cc:194] Registered new tserver with Master: bcfece602314479fa748b61365165f4d (127.12.234.2:43213)
WARNING: no policy specified for kudu/127.12.234.3@KRBTEST.COM; defaulting to no policy
I20260809 04:43:55.692166 16505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.2@KRBTEST.COM'} at 127.12.234.2:37615
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
I20260809 04:43:55.725844 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:0
--local_ip_for_outbound_sockets=127.12.234.3
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:36487
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:35669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:43:56.122597 16700 heartbeater.cc:503] Master 127.12.234.62:36487 was elected leader, sending a full tablet report...
W20260809 04:43:56.131625 16846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:56.132027 16846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:56.132288 16846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:56.142081 16846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:56.142230 16846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:56.142416 16846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:43:56.154125 16846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:35669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:36487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:56.155622 16846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:56.157035 16846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:56.173981 16851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:56.173923 16855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:56.173945 16852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:56.174875 16846 server_base.cc:1034] running on GCE node
I20260809 04:43:56.175593 16846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:56.176757 16846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:56.178032 16846 hybrid_clock.cc:648] HybridClock initialized: now 1786250636177961 us; error 46 us; skew 500 ppm
I20260809 04:43:56.178414 16846 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:56 dist-test-slave-1jjb krb5kdc[16457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250636, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:56.182631 16846 init.cc:377] Logged in from keytab as kudu/127.12.234.3@KRBTEST.COM (short username kudu)
I20260809 04:43:56.184746 16846 webserver.cc:460] Webserver started at http://127.12.234.3:44723/ using document root <none> and password file <none>
I20260809 04:43:56.185628 16846 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:56.185807 16846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:56.186208 16846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:56.190021 16846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/data/instance:
uuid: "9dde2a69c7984a5285e4f4dabbf2db38"
format_stamp: "Formatted at 2026-08-09 04:43:56 on dist-test-slave-1jjb"
server_key: "8dcf278337efd47f2ddbe12b7141eac3"
server_key_iv: "beac9b30ff9f016fff25ebbf521f3199"
server_key_version: "encryptionkey@0"
I20260809 04:43:56.191009 16846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance:
uuid: "9dde2a69c7984a5285e4f4dabbf2db38"
format_stamp: "Formatted at 2026-08-09 04:43:56 on dist-test-slave-1jjb"
server_key: "8dcf278337efd47f2ddbe12b7141eac3"
server_key_iv: "beac9b30ff9f016fff25ebbf521f3199"
server_key_version: "encryptionkey@0"
I20260809 04:43:56.198180 16846 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.004s
I20260809 04:43:56.202893 16862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:56.204499 16846 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 04:43:56.204720 16846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "9dde2a69c7984a5285e4f4dabbf2db38"
format_stamp: "Formatted at 2026-08-09 04:43:56 on dist-test-slave-1jjb"
server_key: "8dcf278337efd47f2ddbe12b7141eac3"
server_key_iv: "beac9b30ff9f016fff25ebbf521f3199"
server_key_version: "encryptionkey@0"
I20260809 04:43:56.204960 16846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:56.226262 16846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:56.232666 16846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:56.233101 16846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:56.234295 16846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:43:56.236008 16846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:43:56.236148 16846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 04:43:56.236364 16846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:43:56.236446 16846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:56.278769 16846 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:45751
I20260809 04:43:56.278877 16979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:45751 every 8 connection(s)
I20260809 04:43:56.280344 16846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:43:56.286248 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 16846
I20260809 04:43:56.286502 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance
I20260809 04:43:56.287079 13224 external_mini_cluster.cc:1428] Setting key a7e50da91dc5fe5507f1cb015b6bc0e9
Aug 09 04:43:56 dist-test-slave-1jjb krb5kdc[16457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250636, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:56.308377 16982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:56.283502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:34387 (local address 127.12.234.62:36487)
0809 04:43:56.284272 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:56.284281 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:56.285812 (+ 1531us) server_negotiation.cc:407] Connection header received
0809 04:43:56.288403 (+ 2591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:56.288409 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:56.288525 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:56.288664 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:56.290796 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:56.291884 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:56.293767 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:56.294234 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:56.294393 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:56.299437 (+ 5044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:56.299445 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:56.299448 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:56.299501 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:56.303803 (+ 4302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:56.305194 (+ 1391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:56.305199 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:56.305202 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:56.305370 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:56.305970 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:56.305975 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:56.305977 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:56.306409 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:56.306560 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:56.307713 (+ 1153us) server_negotiation.cc:299] Negotiation successful
0809 04:43:56.307940 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":331,"threads_started":1}
I20260809 04:43:56.309928 16983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:56.284457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:36487 (local address 127.12.234.3:34387)
0809 04:43:56.285558 (+ 1101us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:56.285615 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:56.287969 (+ 2354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:56.289032 (+ 1063us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:56.289049 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:56.289867 (+ 818us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:56.290587 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:56.290608 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:56.292128 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:56.292134 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:56.293585 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:56.293599 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:56.294510 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:56.294521 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:56.294792 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:56.295714 (+ 922us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:56.295767 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:56.299045 (+ 3278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:56.304066 (+ 5021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:56.304080 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:56.304131 (+ 51us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:56.305020 (+ 889us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:56.305551 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:56.305555 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:56.305558 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:56.305837 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:56.306635 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:56.306645 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:56.307233 (+ 588us) client_negotiation.cc:769] Sending connection context
0809 04:43:56.307594 (+ 361us) client_negotiation.cc:240] Negotiation successful
0809 04:43:56.308297 (+ 703us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"thread_start_us":428,"threads_started":1}
I20260809 04:43:56.312523 16980 heartbeater.cc:347] Connected to a master server at 127.12.234.62:36487
I20260809 04:43:56.313104 16980 heartbeater.cc:464] Registering TS with master...
I20260809 04:43:56.314137 16980 heartbeater.cc:511] Master 127.12.234.62:36487 requested a full tablet report, sending...
I20260809 04:43:56.316406 16505 ts_manager.cc:194] Registered new tserver with Master: 9dde2a69c7984a5285e4f4dabbf2db38 (127.12.234.3:45751)
I20260809 04:43:56.317562 16505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.3@KRBTEST.COM'} at 127.12.234.3:34387
I20260809 04:43:56.318799 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 04:43:56 dist-test-slave-1jjb krb5kdc[16457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250636, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 04:43:56 dist-test-slave-1jjb krb5kdc[16457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250636, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:56.385623 16982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:56.370583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58666 (local address 127.12.234.62:36487)
0809 04:43:56.370760 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:56.370765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:56.371300 (+ 535us) server_negotiation.cc:407] Connection header received
0809 04:43:56.371469 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:56.371473 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:56.371552 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:56.371634 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:56.373107 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:56.373858 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:56.375142 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:56.375593 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:56.375691 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:56.379622 (+ 3931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:56.379628 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:56.379631 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:56.379670 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:56.382262 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:56.383087 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:56.383093 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:56.383098 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:56.383275 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:56.383987 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:56.383992 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:56.383996 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:56.384484 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:56.384732 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:56.385030 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 04:43:56.385281 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 04:43:56.386341 16504 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58666
I20260809 04:43:56.388487 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 16568
Aug 09 04:43:56 dist-test-slave-1jjb krb5kdc[16457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250636, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.12.234.1@KRBTEST.COM
I20260809 04:43:56.406530 16992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:56.390560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50166 (local address 127.12.234.1:42875)
0809 04:43:56.391278 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:56.391287 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:56.391315 (+ 28us) server_negotiation.cc:407] Connection header received
0809 04:43:56.391383 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:56.391391 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:56.391763 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:56.392005 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:56.393413 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:56.394582 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:56.395876 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:56.396666 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:56.396812 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:56.400940 (+ 4128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:56.400952 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:56.400969 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:56.401003 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:56.403747 (+ 2744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:56.404502 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:56.404509 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:56.404511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:56.404635 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:56.405018 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:56.405021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:56.405023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:56.405453 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:56.405702 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:56.405982 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 04:43:56.406191 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":332,"threads_started":1}
W20260809 04:43:56.407877 16695 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50166
W20260809 04:43:56.409111 13224 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 16568: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:43:56.409340 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 16568
I20260809 04:43:56.428589 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 16706
Aug 09 04:43:56 dist-test-slave-1jjb krb5kdc[16457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250636, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.12.234.2@KRBTEST.COM
I20260809 04:43:56.444767 16993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:56.429429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36580 (local address 127.12.234.2:43213)
0809 04:43:56.429980 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:56.429988 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:56.430013 (+ 25us) server_negotiation.cc:407] Connection header received
0809 04:43:56.430077 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:56.430084 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:56.430297 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:56.430476 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:56.431737 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:56.432827 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:56.434035 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:56.434610 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:56.434754 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:56.439052 (+ 4298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:56.439070 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:56.439092 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:56.439135 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:56.442188 (+ 3053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:56.442899 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:56.442906 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:56.442908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:56.443023 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:56.443326 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:56.443329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:56.443331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:56.443687 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:56.443914 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:56.444246 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 04:43:56.444455 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":278,"threads_started":1}
W20260809 04:43:56.446055 16835 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36580
W20260809 04:43:56.447165 13224 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 16706: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:43:56.447324 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 16706
I20260809 04:43:56.466342 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 16846
Aug 09 04:43:56 dist-test-slave-1jjb krb5kdc[16457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250636, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.12.234.3@KRBTEST.COM
I20260809 04:43:56.482782 16994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:56.467285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52844 (local address 127.12.234.3:45751)
0809 04:43:56.467874 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:56.467881 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:56.467911 (+ 30us) server_negotiation.cc:407] Connection header received
0809 04:43:56.467973 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:56.467979 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:56.468286 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:56.468455 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:56.469810 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:56.470930 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:56.472249 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:56.472799 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:56.472947 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:56.477257 (+ 4310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:56.477269 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:56.477285 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:56.477320 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:56.479931 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:56.480653 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:56.480660 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:56.480663 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:56.480781 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:56.481167 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:56.481171 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:56.481173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:56.481509 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:56.481840 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:56.482154 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 04:43:56.482434 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":274,"threads_started":1}
W20260809 04:43:56.484062 16974 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52844
W20260809 04:43:56.485200 13224 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 16846: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:43:56.485368 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 16846
I20260809 04:43:56.512050 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 16471
I20260809 04:43:56.513316 16535 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:43:56.629916 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 16471
2026-08-09T04:43:56Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4220 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:43:56 dist-test-slave-1jjb krb5kdc[16998](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:43:56 dist-test-slave-1jjb krb5kdc[16998](info): set up 2 sockets
Aug 09 04:43:56 dist-test-slave-1jjb krb5kdc[16998](info): commencing operation
krb5kdc: starting...
W20260809 04:43:57.827039 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:43:57 dist-test-slave-1jjb krb5kdc[16998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:43:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:43:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:43:58.051609 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:42057
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:46083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:42057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:58.406804 17011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:58.407125 17011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:58.407220 17011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:58.414682 17011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:43:58.414793 17011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:58.414839 17011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:58.414899 17011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:43:58.414944 17011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:43:58.424990 17011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:46083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:42057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:42057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:58.426301 17011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:58.427598 17011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:58.439589 17019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:58.439517 17016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:58.439625 17017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:58.440930 17011 server_base.cc:1034] running on GCE node
I20260809 04:43:58.441721 17011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:58.443256 17011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:58.444461 17011 hybrid_clock.cc:648] HybridClock initialized: now 1786250638444422 us; error 29 us; skew 500 ppm
I20260809 04:43:58.444829 17011 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:58 dist-test-slave-1jjb krb5kdc[16998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250638, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:58.451043 17011 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:43:58.452564 17011 webserver.cc:460] Webserver started at http://127.12.234.62:35139/ using document root <none> and password file <none>
I20260809 04:43:58.453279 17011 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:58.453480 17011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:58.453838 17011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:58.456904 17011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "16c5d71989f241ffb2f5d7756225e82d"
format_stamp: "Formatted at 2026-08-09 04:43:58 on dist-test-slave-1jjb"
server_key: "03ea065e3eaed8606ba4444b8083d098"
server_key_iv: "690b17d41cd1c738dc7af6a309f07c98"
server_key_version: "encryptionkey@0"
I20260809 04:43:58.457703 17011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "16c5d71989f241ffb2f5d7756225e82d"
format_stamp: "Formatted at 2026-08-09 04:43:58 on dist-test-slave-1jjb"
server_key: "03ea065e3eaed8606ba4444b8083d098"
server_key_iv: "690b17d41cd1c738dc7af6a309f07c98"
server_key_version: "encryptionkey@0"
I20260809 04:43:58.462980 17011 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 04:43:58.466418 17026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:58.467748 17011 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 04:43:58.467942 17011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "16c5d71989f241ffb2f5d7756225e82d"
format_stamp: "Formatted at 2026-08-09 04:43:58 on dist-test-slave-1jjb"
server_key: "03ea065e3eaed8606ba4444b8083d098"
server_key_iv: "690b17d41cd1c738dc7af6a309f07c98"
server_key_version: "encryptionkey@0"
I20260809 04:43:58.468196 17011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:58.482710 17011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:58.487082 17011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:58.487373 17011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:58.506040 17011 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:42057
I20260809 04:43:58.506088 17080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:42057 every 8 connection(s)
I20260809 04:43:58.507922 17011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:43:58.511909 17081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:43:58.513645 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 17011
I20260809 04:43:58.513921 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:43:58.514935 13224 external_mini_cluster.cc:1428] Setting key 29c02c741484f24a418e6e61aaa9fab2
I20260809 04:43:58.527575 17081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16c5d71989f241ffb2f5d7756225e82d: Bootstrap starting.
Aug 09 04:43:58 dist-test-slave-1jjb krb5kdc[16998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:58.531319 17081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16c5d71989f241ffb2f5d7756225e82d: Neither blocks nor log segments found. Creating new log.
I20260809 04:43:58.532438 17081 log.cc:824] T 00000000000000000000000000000000 P 16c5d71989f241ffb2f5d7756225e82d: Log is configured to *not* fsync() on all Append() calls
I20260809 04:43:58.535362 17081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16c5d71989f241ffb2f5d7756225e82d: No bootstrap required, opened a new log
I20260809 04:43:58.538239 17084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:58.517094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50386 (local address 127.12.234.62:42057)
0809 04:43:58.517910 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:58.517921 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:58.517970 (+ 49us) server_negotiation.cc:407] Connection header received
0809 04:43:58.519540 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:58.519562 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:58.519940 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:58.520330 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:58.522343 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:58.523634 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:58.525566 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:58.526231 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:58.526357 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:58.530938 (+ 4581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:58.530945 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:58.530960 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:58.530994 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:58.534267 (+ 3273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:58.534954 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:58.534960 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:58.534966 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:58.535129 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:58.535499 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:58.535511 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:58.535514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:58.536028 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:58.536393 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:58.536830 (+ 437us) server_negotiation.cc:299] Negotiation successful
0809 04:43:58.537265 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":382,"threads_started":1}
I20260809 04:43:58.543315 17081 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16c5d71989f241ffb2f5d7756225e82d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c5d71989f241ffb2f5d7756225e82d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42057 } }
I20260809 04:43:58.543788 17081 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16c5d71989f241ffb2f5d7756225e82d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:43:58.543932 17081 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16c5d71989f241ffb2f5d7756225e82d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16c5d71989f241ffb2f5d7756225e82d, State: Initialized, Role: FOLLOWER
I20260809 04:43:58.544587 17081 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16c5d71989f241ffb2f5d7756225e82d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c5d71989f241ffb2f5d7756225e82d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42057 } }
I20260809 04:43:58.544838 17081 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16c5d71989f241ffb2f5d7756225e82d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:43:58.544996 17081 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16c5d71989f241ffb2f5d7756225e82d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:43:58.545187 17081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16c5d71989f241ffb2f5d7756225e82d [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:43:58.547255 17081 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16c5d71989f241ffb2f5d7756225e82d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c5d71989f241ffb2f5d7756225e82d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42057 } }
I20260809 04:43:58.547739 17081 leader_election.cc:302] T 00000000000000000000000000000000 P 16c5d71989f241ffb2f5d7756225e82d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16c5d71989f241ffb2f5d7756225e82d; no voters:
I20260809 04:43:58.548357 17081 leader_election.cc:288] T 00000000000000000000000000000000 P 16c5d71989f241ffb2f5d7756225e82d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:43:58.548512 17086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16c5d71989f241ffb2f5d7756225e82d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:43:58.549198 17086 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16c5d71989f241ffb2f5d7756225e82d [term 1 LEADER]: Becoming Leader. State: Replica: 16c5d71989f241ffb2f5d7756225e82d, State: Running, Role: LEADER
I20260809 04:43:58.549695 17086 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16c5d71989f241ffb2f5d7756225e82d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c5d71989f241ffb2f5d7756225e82d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42057 } }
I20260809 04:43:58.550356 17081 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16c5d71989f241ffb2f5d7756225e82d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:43:58.553340 17087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16c5d71989f241ffb2f5d7756225e82d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16c5d71989f241ffb2f5d7756225e82d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c5d71989f241ffb2f5d7756225e82d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42057 } } }
I20260809 04:43:58.553815 17087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16c5d71989f241ffb2f5d7756225e82d [sys.catalog]: This master's current role is: LEADER
I20260809 04:43:58.554813 17088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16c5d71989f241ffb2f5d7756225e82d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16c5d71989f241ffb2f5d7756225e82d. Latest consensus state: current_term: 1 leader_uuid: "16c5d71989f241ffb2f5d7756225e82d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16c5d71989f241ffb2f5d7756225e82d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42057 } } }
I20260809 04:43:58.555133 17088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16c5d71989f241ffb2f5d7756225e82d [sys.catalog]: This master's current role is: LEADER
I20260809 04:43:58.556857 17097 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:43:58.561954 17097 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:43:58.569947 17097 catalog_manager.cc:1347] Generated new cluster ID: 4996d44af43244dfb688cdef7f3789b9
I20260809 04:43:58.570071 17097 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:43:58.594714 17097 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:43:58.596123 17097 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:43:58.606890 17097 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16c5d71989f241ffb2f5d7756225e82d: Generated new TSK 0
I20260809 04:43:58.607731 17097 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
I20260809 04:43:58.672345 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:0
--local_ip_for_outbound_sockets=127.12.234.1
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:42057
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:46083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:59.067600 17109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:59.068025 17109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:59.068215 17109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:59.075949 17109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:59.076148 17109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:59.076318 17109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:43:59.088071 17109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:46083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:42057
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:59.089670 17109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:59.091099 17109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:59.108162 17114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:59.108137 17115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:59.108469 17117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:59.109781 17109 server_base.cc:1034] running on GCE node
I20260809 04:43:59.110651 17109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:59.112268 17109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:59.113565 17109 hybrid_clock.cc:648] HybridClock initialized: now 1786250639113490 us; error 60 us; skew 500 ppm
I20260809 04:43:59.113962 17109 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:59 dist-test-slave-1jjb krb5kdc[16998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250639, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:59.118531 17109 init.cc:377] Logged in from keytab as kudu/127.12.234.1@KRBTEST.COM (short username kudu)
I20260809 04:43:59.121269 17109 webserver.cc:460] Webserver started at http://127.12.234.1:33833/ using document root <none> and password file <none>
I20260809 04:43:59.122475 17109 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:59.122694 17109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:59.123386 17109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:59.128196 17109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-0/data/instance:
uuid: "e5747fcc637d4a71841f565c9b179283"
format_stamp: "Formatted at 2026-08-09 04:43:59 on dist-test-slave-1jjb"
server_key: "b7a8629a7081764a72a8ab8bd17a46e7"
server_key_iv: "eef3282006698e1c57619a847027f797"
server_key_version: "encryptionkey@0"
I20260809 04:43:59.129783 17109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance:
uuid: "e5747fcc637d4a71841f565c9b179283"
format_stamp: "Formatted at 2026-08-09 04:43:59 on dist-test-slave-1jjb"
server_key: "b7a8629a7081764a72a8ab8bd17a46e7"
server_key_iv: "eef3282006698e1c57619a847027f797"
server_key_version: "encryptionkey@0"
I20260809 04:43:59.137085 17109 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260809 04:43:59.142462 17125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:59.144850 17109 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260809 04:43:59.145125 17109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "e5747fcc637d4a71841f565c9b179283"
format_stamp: "Formatted at 2026-08-09 04:43:59 on dist-test-slave-1jjb"
server_key: "b7a8629a7081764a72a8ab8bd17a46e7"
server_key_iv: "eef3282006698e1c57619a847027f797"
server_key_version: "encryptionkey@0"
I20260809 04:43:59.145459 17109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:59.168486 17109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:59.174067 17109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:59.174418 17109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:59.175628 17109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:43:59.177453 17109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:43:59.177544 17109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:59.177678 17109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:43:59.177753 17109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:59.218029 17109 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:46237
I20260809 04:43:59.218063 17242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:46237 every 8 connection(s)
I20260809 04:43:59.219488 17109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:43:59.221724 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 17109
I20260809 04:43:59.221932 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance
I20260809 04:43:59.222657 13224 external_mini_cluster.cc:1428] Setting key 9d8248b05aab5c60588281a1fb506ccd
Aug 09 04:43:59 dist-test-slave-1jjb krb5kdc[16998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250639, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:59.248431 17245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:59.222727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:42307 (local address 127.12.234.62:42057)
0809 04:43:59.223228 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:59.223235 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:59.224363 (+ 1128us) server_negotiation.cc:407] Connection header received
0809 04:43:59.227290 (+ 2927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:59.227294 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:59.227362 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:59.227478 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:59.229855 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:59.230619 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:59.233986 (+ 3367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:59.234462 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:59.234606 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:59.240104 (+ 5498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:59.240115 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:59.240123 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:59.240174 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:59.243697 (+ 3523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:59.244877 (+ 1180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:59.244883 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:59.244886 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:59.245040 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:59.245572 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:59.245583 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:59.245586 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:59.245942 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:59.246336 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:59.247730 (+ 1394us) server_negotiation.cc:299] Negotiation successful
0809 04:43:59.247988 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":251,"threads_started":1}
I20260809 04:43:59.250161 17246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:59.223375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:42057 (local address 127.12.234.1:42307)
0809 04:43:59.224215 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:59.224263 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:59.226935 (+ 2672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:59.227777 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:59.227793 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:59.228691 (+ 898us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:59.229627 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:59.229657 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:59.230812 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:59.230821 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:59.233653 (+ 2832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:59.233692 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:59.234866 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:59.234877 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:59.235211 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:59.236623 (+ 1412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:59.236667 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:59.239701 (+ 3034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:59.243959 (+ 4258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:59.243982 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:59.244006 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:59.244696 (+ 690us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:59.245180 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:59.245185 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:59.245189 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:59.245432 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:59.246275 (+ 843us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:59.246287 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:59.247045 (+ 758us) client_negotiation.cc:769] Sending connection context
0809 04:43:59.247513 (+ 468us) client_negotiation.cc:240] Negotiation successful
0809 04:43:59.248342 (+ 829us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":314,"threads_started":1}
WARNING: no policy specified for kudu/127.12.234.2@KRBTEST.COM; defaulting to no policy
I20260809 04:43:59.252611 17243 heartbeater.cc:347] Connected to a master server at 127.12.234.62:42057
I20260809 04:43:59.253041 17243 heartbeater.cc:464] Registering TS with master...
I20260809 04:43:59.254137 17243 heartbeater.cc:511] Master 127.12.234.62:42057 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.2@KRBTEST.COM" created.
I20260809 04:43:59.258267 17043 ts_manager.cc:194] Registered new tserver with Master: e5747fcc637d4a71841f565c9b179283 (127.12.234.1:46237)
I20260809 04:43:59.260743 17043 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.1@KRBTEST.COM'} at 127.12.234.1:42307
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
I20260809 04:43:59.287575 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:0
--local_ip_for_outbound_sockets=127.12.234.2
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:42057
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:46083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:43:59.694655 17249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:43:59.695035 17249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:43:59.695210 17249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:43:59.702773 17249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:43:59.702936 17249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:43:59.703123 17249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:43:59.713387 17249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:46083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:42057
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:43:59.714850 17249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:43:59.716360 17249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:43:59.730535 17254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:59.730506 17255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:43:59.730705 17257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:43:59.731292 17249 server_base.cc:1034] running on GCE node
I20260809 04:43:59.731989 17249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:43:59.733084 17249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:43:59.734272 17249 hybrid_clock.cc:648] HybridClock initialized: now 1786250639734228 us; error 32 us; skew 500 ppm
I20260809 04:43:59.734624 17249 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:43:59 dist-test-slave-1jjb krb5kdc[16998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250639, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:43:59.737849 17249 init.cc:377] Logged in from keytab as kudu/127.12.234.2@KRBTEST.COM (short username kudu)
I20260809 04:43:59.739329 17249 webserver.cc:460] Webserver started at http://127.12.234.2:40497/ using document root <none> and password file <none>
I20260809 04:43:59.740044 17249 fs_manager.cc:359] Metadata directory not provided
I20260809 04:43:59.740224 17249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:43:59.740525 17249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:43:59.743709 17249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-1/data/instance:
uuid: "03aa5aa20a454122855db0157eb82f95"
format_stamp: "Formatted at 2026-08-09 04:43:59 on dist-test-slave-1jjb"
server_key: "fe0719a98b12c55fb77b59f01b6757ef"
server_key_iv: "540be9edb3f4fbfc420b0d66271fc8dc"
server_key_version: "encryptionkey@0"
I20260809 04:43:59.744622 17249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance:
uuid: "03aa5aa20a454122855db0157eb82f95"
format_stamp: "Formatted at 2026-08-09 04:43:59 on dist-test-slave-1jjb"
server_key: "fe0719a98b12c55fb77b59f01b6757ef"
server_key_iv: "540be9edb3f4fbfc420b0d66271fc8dc"
server_key_version: "encryptionkey@0"
I20260809 04:43:59.750102 17249 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 04:43:59.754138 17264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:59.755529 17249 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260809 04:43:59.755719 17249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "03aa5aa20a454122855db0157eb82f95"
format_stamp: "Formatted at 2026-08-09 04:43:59 on dist-test-slave-1jjb"
server_key: "fe0719a98b12c55fb77b59f01b6757ef"
server_key_iv: "540be9edb3f4fbfc420b0d66271fc8dc"
server_key_version: "encryptionkey@0"
I20260809 04:43:59.755939 17249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:43:59.775766 17249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:43:59.780138 17249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:43:59.780444 17249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:43:59.781471 17249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:43:59.782982 17249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:43:59.783092 17249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:59.783255 17249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:43:59.783330 17249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:43:59.820632 17249 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:43623
I20260809 04:43:59.820730 17380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:43623 every 8 connection(s)
I20260809 04:43:59.822144 17249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:43:59.826213 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 17249
I20260809 04:43:59.826442 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance
I20260809 04:43:59.827077 13224 external_mini_cluster.cc:1428] Setting key d42d3383a138ef759d5173da314d7dc5
Aug 09 04:43:59 dist-test-slave-1jjb krb5kdc[16998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250639, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:43:59.845569 17383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:59.825765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:55879 (local address 127.12.234.62:42057)
0809 04:43:59.826320 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0809 04:43:59.826328 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:43:59.827499 (+ 1171us) server_negotiation.cc:407] Connection header received
0809 04:43:59.829826 (+ 2327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:43:59.829832 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:43:59.829937 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:43:59.830062 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:43:59.831618 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:59.832435 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:59.833998 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:43:59.834450 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:43:59.834593 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:59.838533 (+ 3940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:43:59.838544 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:43:59.838547 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:43:59.838590 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:43:59.841319 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:59.842527 (+ 1208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:59.842535 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:59.842538 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:59.842782 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:43:59.843439 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:43:59.843444 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:43:59.843482 (+ 38us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:43:59.843808 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:43:59.843936 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 04:43:59.845020 (+ 1084us) server_negotiation.cc:299] Negotiation successful
0809 04:43:59.845250 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":296,"threads_started":1}
I20260809 04:43:59.847460 17384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:43:59.826325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:42057 (local address 127.12.234.2:55879)
0809 04:43:59.827304 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0809 04:43:59.827360 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 04:43:59.829534 (+ 2174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:43:59.830338 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:43:59.830347 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:43:59.830865 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:43:59.831445 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:59.831462 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:59.832600 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:59.832605 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:59.833846 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:43:59.833859 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:43:59.834712 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:43:59.834721 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:43:59.834951 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:43:59.835671 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:43:59.835698 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:43:59.838249 (+ 2551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:43:59.841472 (+ 3223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:59.841485 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:59.841507 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:59.842256 (+ 749us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:59.842962 (+ 706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:43:59.842970 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:43:59.842976 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:43:59.843244 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:43:59.843967 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:43:59.843979 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:43:59.844546 (+ 567us) client_negotiation.cc:769] Sending connection context
0809 04:43:59.844939 (+ 393us) client_negotiation.cc:240] Negotiation successful
0809 04:43:59.845587 (+ 648us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":365,"threads_started":1}
I20260809 04:43:59.849339 17381 heartbeater.cc:347] Connected to a master server at 127.12.234.62:42057
I20260809 04:43:59.849771 17381 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.12.234.3@KRBTEST.COM; defaulting to no policy
I20260809 04:43:59.851140 17381 heartbeater.cc:511] Master 127.12.234.62:42057 requested a full tablet report, sending...
I20260809 04:43:59.853195 17043 ts_manager.cc:194] Registered new tserver with Master: 03aa5aa20a454122855db0157eb82f95 (127.12.234.2:43623)
I20260809 04:43:59.854156 17043 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.2@KRBTEST.COM'} at 127.12.234.2:55879
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
I20260809 04:43:59.883605 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:0
--local_ip_for_outbound_sockets=127.12.234.3
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:42057
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:46083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:00.245999 17387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:00.246424 17387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:00.246626 17387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:00.254567 17387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:00.254674 17387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:00.254792 17387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:44:00.264956 17387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:46083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:42057
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:00.265261 17243 heartbeater.cc:503] Master 127.12.234.62:42057 was elected leader, sending a full tablet report...
I20260809 04:44:00.266553 17387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:00.268167 17387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:00.281517 17392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:00.281546 17395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:00.281579 17393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:00.282932 17387 server_base.cc:1034] running on GCE node
I20260809 04:44:00.283643 17387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:00.284760 17387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:00.285975 17387 hybrid_clock.cc:648] HybridClock initialized: now 1786250640285903 us; error 55 us; skew 500 ppm
I20260809 04:44:00.286326 17387 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:00 dist-test-slave-1jjb krb5kdc[16998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250640, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:00.289600 17387 init.cc:377] Logged in from keytab as kudu/127.12.234.3@KRBTEST.COM (short username kudu)
I20260809 04:44:00.291136 17387 webserver.cc:460] Webserver started at http://127.12.234.3:43823/ using document root <none> and password file <none>
I20260809 04:44:00.291811 17387 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:00.291926 17387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:00.292294 17387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:00.295253 17387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-2/data/instance:
uuid: "86257378d2af4afb83a629adb3f5d7e7"
format_stamp: "Formatted at 2026-08-09 04:44:00 on dist-test-slave-1jjb"
server_key: "0ea7cde4a7fef80bd34d441329d241a1"
server_key_iv: "d132a5b225cdb479adc8ab66a08a43a9"
server_key_version: "encryptionkey@0"
I20260809 04:44:00.296041 17387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance:
uuid: "86257378d2af4afb83a629adb3f5d7e7"
format_stamp: "Formatted at 2026-08-09 04:44:00 on dist-test-slave-1jjb"
server_key: "0ea7cde4a7fef80bd34d441329d241a1"
server_key_iv: "d132a5b225cdb479adc8ab66a08a43a9"
server_key_version: "encryptionkey@0"
I20260809 04:44:00.301476 17387 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 04:44:00.305022 17403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:00.306403 17387 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260809 04:44:00.306597 17387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "86257378d2af4afb83a629adb3f5d7e7"
format_stamp: "Formatted at 2026-08-09 04:44:00 on dist-test-slave-1jjb"
server_key: "0ea7cde4a7fef80bd34d441329d241a1"
server_key_iv: "d132a5b225cdb479adc8ab66a08a43a9"
server_key_version: "encryptionkey@0"
I20260809 04:44:00.306771 17387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:00.324841 17387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:00.329361 17387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:00.329667 17387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:00.330647 17387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:00.332051 17387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:00.332190 17387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:00.332350 17387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:00.332424 17387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:00.367422 17387 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:34437
I20260809 04:44:00.367465 17518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:34437 every 8 connection(s)
I20260809 04:44:00.368851 17387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:44:00.378258 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 17387
I20260809 04:44:00.378448 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance
I20260809 04:44:00.378893 13224 external_mini_cluster.cc:1428] Setting key 248de7ce8dd4d221f9676e3903f86b8b
Aug 09 04:44:00 dist-test-slave-1jjb krb5kdc[16998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250640, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:44:00.388743 17521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:00.371592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:57521 (local address 127.12.234.62:42057)
0809 04:44:00.372142 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:00.372148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:00.373091 (+ 943us) server_negotiation.cc:407] Connection header received
0809 04:44:00.374878 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:00.374883 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:00.374946 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:00.375046 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:00.376533 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:00.377295 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:00.378860 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:00.379428 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:00.379517 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:00.383550 (+ 4033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:00.383555 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:00.383558 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:00.383587 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:00.386283 (+ 2696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:00.386887 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:00.386890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:00.386892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:00.386997 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:00.387352 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:00.387355 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:00.387356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:00.387600 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:00.387710 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:00.388329 (+ 619us) server_negotiation.cc:299] Negotiation successful
0809 04:44:00.388474 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":257,"threads_started":1}
I20260809 04:44:00.389665 17522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:00.372055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:42057 (local address 127.12.234.3:57521)
0809 04:44:00.372885 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:00.372934 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:00.374632 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:00.375239 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:00.375246 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:00.375754 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:00.376389 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:00.376405 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:00.377437 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:00.377440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:00.378734 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:00.378746 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:00.379670 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:00.379676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:00.379855 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:00.380639 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:00.380664 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:00.383329 (+ 2665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:00.386395 (+ 3066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:00.386401 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:00.386420 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:00.386787 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:00.387080 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:00.387083 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:00.387085 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:00.387257 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:00.387701 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:00.387706 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:00.387970 (+ 264us) client_negotiation.cc:769] Sending connection context
0809 04:44:00.388232 (+ 262us) client_negotiation.cc:240] Negotiation successful
0809 04:44:00.388573 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":350,"threads_started":1}
I20260809 04:44:00.391021 17519 heartbeater.cc:347] Connected to a master server at 127.12.234.62:42057
I20260809 04:44:00.391335 17519 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:00.392020 17519 heartbeater.cc:511] Master 127.12.234.62:42057 requested a full tablet report, sending...
I20260809 04:44:00.393795 17043 ts_manager.cc:194] Registered new tserver with Master: 86257378d2af4afb83a629adb3f5d7e7 (127.12.234.3:34437)
I20260809 04:44:00.394727 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:44:00.394919 17043 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.3@KRBTEST.COM'} at 127.12.234.3:57521
Aug 09 04:44:00 dist-test-slave-1jjb krb5kdc[16998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 04:44:00 dist-test-slave-1jjb krb5kdc[16998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.1@KRBTEST.COM
I20260809 04:44:00.447117 17527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:00.432965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41554 (local address 127.12.234.1:46237)
0809 04:44:00.433587 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:00.433594 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:00.433621 (+ 27us) server_negotiation.cc:407] Connection header received
0809 04:44:00.433691 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:00.433697 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:00.433910 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:00.434063 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:00.435418 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:00.436500 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:00.437632 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:00.438179 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:00.438276 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:00.442179 (+ 3903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:00.442189 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:00.442203 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:00.442230 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:00.444815 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:00.445407 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:00.445414 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:00.445416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:00.445529 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:00.445863 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:00.445866 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:00.445868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:00.446194 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:00.446415 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:00.446666 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 04:44:00.446840 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":300,"threads_started":1}
Aug 09 04:44:00 dist-test-slave-1jjb krb5kdc[16998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:44:00.465893 17521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:00.451913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50390 (local address 127.12.234.62:42057)
0809 04:44:00.452140 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:00.452146 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:00.452290 (+ 144us) server_negotiation.cc:407] Connection header received
0809 04:44:00.452478 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:00.452484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:00.452566 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:00.452667 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:00.453958 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:00.454708 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:00.456011 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:00.456659 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:00.456803 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:00.460857 (+ 4054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:00.460863 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:00.460866 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:00.460893 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:00.463200 (+ 2307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:00.463827 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:00.463833 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:00.463836 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:00.463961 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:00.464406 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:00.464411 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:00.464414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:00.464738 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:00.465022 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:00.465342 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 04:44:00.465577 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 04:44:00.466444 17043 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50390
I20260809 04:44:00.467470 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 17109
I20260809 04:44:00.483706 17527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:00.468387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41570 (local address 127.12.234.1:46237)
0809 04:44:00.468544 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:00.468549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:00.468888 (+ 339us) server_negotiation.cc:407] Connection header received
0809 04:44:00.469010 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:00.469014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:00.469089 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:00.469166 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:00.470286 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:00.470985 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:00.473008 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:00.473587 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:00.473729 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:00.476712 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:00.476717 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:00.476720 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:00.476749 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:00.480472 (+ 3723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:00.481266 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:00.481271 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:00.481276 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:00.481423 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:00.481851 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:00.481856 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:00.481859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:00.482283 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:00.482563 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:00.482979 (+ 416us) server_negotiation.cc:299] Negotiation successful
0809 04:44:00.483261 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 04:44:00.484396 17237 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:00.571929 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 17109
I20260809 04:44:00.592188 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 17249
Aug 09 04:44:00 dist-test-slave-1jjb krb5kdc[16998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.2@KRBTEST.COM
I20260809 04:44:00.608433 17532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:00.593180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44828 (local address 127.12.234.2:43623)
0809 04:44:00.593729 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:00.593746 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:00.593773 (+ 27us) server_negotiation.cc:407] Connection header received
0809 04:44:00.593835 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:00.593841 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:00.594054 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:00.594181 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:00.595363 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:00.596409 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:00.597631 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:00.598147 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:00.598242 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:00.602449 (+ 4207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:00.602460 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:00.602474 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:00.602505 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:00.605601 (+ 3096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:00.606273 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:00.606281 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:00.606283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:00.606409 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:00.606803 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:00.606808 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:00.606811 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:00.607144 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:00.607460 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:00.607787 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 04:44:00.608050 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":295,"threads_started":1}
I20260809 04:44:00.609670 17376 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:00.690598 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 17249
I20260809 04:44:00.709455 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 17387
Aug 09 04:44:00 dist-test-slave-1jjb krb5kdc[16998](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.3@KRBTEST.COM
I20260809 04:44:00.725008 17534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:00.710439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59010 (local address 127.12.234.3:34437)
0809 04:44:00.711077 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:00.711084 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:00.711113 (+ 29us) server_negotiation.cc:407] Connection header received
0809 04:44:00.711172 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:00.711178 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:00.711384 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:00.711583 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:00.712882 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:00.713871 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:00.715186 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:00.715740 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:00.715884 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:00.720128 (+ 4244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:00.720140 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:00.720154 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:00.720184 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:00.722611 (+ 2427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:00.723237 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:00.723243 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:00.723245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:00.723355 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:00.723663 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:00.723666 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:00.723668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:00.723973 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:00.724297 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:00.724522 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 04:44:00.724706 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":325,"threads_started":1}
I20260809 04:44:00.726291 17514 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:00.809459 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 17387
I20260809 04:44:00.830569 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 17011
I20260809 04:44:00.831776 17075 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:00.932739 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 17011
2026-08-09T04:44:00Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4300 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:01 dist-test-slave-1jjb krb5kdc[17539](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:01 dist-test-slave-1jjb krb5kdc[17539](info): set up 2 sockets
Aug 09 04:44:01 dist-test-slave-1jjb krb5kdc[17539](info): commencing operation
krb5kdc: starting...
W20260809 04:44:02.079272 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:44:02 dist-test-slave-1jjb krb5kdc[17539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:44:02.328840 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:41207
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:40295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:41207
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:02.676311 17553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:02.676735 17553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:02.676894 17553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:02.685257 17553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:02.685549 17553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:02.685684 17553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:02.685801 17553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:02.685894 17553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:02.697402 17553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:40295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:41207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:41207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:02.699716 17553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:02.701501 17553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:02.716172 17558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:02.716150 17559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:02.716339 17561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:02.716737 17553 server_base.cc:1034] running on GCE node
I20260809 04:44:02.717968 17553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:02.719503 17553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:02.720902 17553 hybrid_clock.cc:648] HybridClock initialized: now 1786250642720848 us; error 49 us; skew 500 ppm
I20260809 04:44:02.721360 17553 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:02 dist-test-slave-1jjb krb5kdc[17539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250642, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:02.725370 17553 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:44:02.726883 17553 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:02.727053 17553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:02.727438 17553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:02.731010 17553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "d486f07c9efd45de9596f2413265c847"
format_stamp: "Formatted at 2026-08-09 04:44:02 on dist-test-slave-1jjb"
server_key: "ba7d342ae06e80f2b9a073a44a6824c4"
server_key_iv: "8a431e47c28bd732a74fdc08aecbf5db"
server_key_version: "encryptionkey@0"
I20260809 04:44:02.732064 17553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "d486f07c9efd45de9596f2413265c847"
format_stamp: "Formatted at 2026-08-09 04:44:02 on dist-test-slave-1jjb"
server_key: "ba7d342ae06e80f2b9a073a44a6824c4"
server_key_iv: "8a431e47c28bd732a74fdc08aecbf5db"
server_key_version: "encryptionkey@0"
I20260809 04:44:02.738217 17553 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260809 04:44:02.742043 17567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:02.743427 17553 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:44:02.743631 17553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "d486f07c9efd45de9596f2413265c847"
format_stamp: "Formatted at 2026-08-09 04:44:02 on dist-test-slave-1jjb"
server_key: "ba7d342ae06e80f2b9a073a44a6824c4"
server_key_iv: "8a431e47c28bd732a74fdc08aecbf5db"
server_key_version: "encryptionkey@0"
I20260809 04:44:02.743875 17553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:02.761780 17553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:02.766433 17553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:02.766726 17553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:02.785946 17553 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:41207
I20260809 04:44:02.785970 17621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:41207 every 8 connection(s)
I20260809 04:44:02.786917 17553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:02.791072 17623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:02.793857 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 17553
I20260809 04:44:02.794098 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:44:02.794795 13224 external_mini_cluster.cc:1428] Setting key 90571e00ca44aad8938a598e60420eee
I20260809 04:44:02.805539 17623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d486f07c9efd45de9596f2413265c847: Bootstrap starting.
I20260809 04:44:02.809638 17623 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d486f07c9efd45de9596f2413265c847: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:02.811059 17623 log.cc:824] T 00000000000000000000000000000000 P d486f07c9efd45de9596f2413265c847: Log is configured to *not* fsync() on all Append() calls
Aug 09 04:44:02 dist-test-slave-1jjb krb5kdc[17539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:44:02.814294 17623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d486f07c9efd45de9596f2413265c847: No bootstrap required, opened a new log
I20260809 04:44:02.820452 17626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:02.797259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50554 (local address 127.12.234.62:41207)
0809 04:44:02.798227 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:02.798244 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:02.798299 (+ 55us) server_negotiation.cc:407] Connection header received
0809 04:44:02.800878 (+ 2579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:02.800919 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:02.801521 (+ 602us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:02.802102 (+ 581us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:02.803579 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:02.805727 (+ 2148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:02.807130 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:02.808317 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:02.808505 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:02.812809 (+ 4304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:02.812818 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:02.812834 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:02.812874 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:02.815877 (+ 3003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:02.816784 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:02.816795 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:02.816802 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:02.817005 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:02.817434 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:02.817438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:02.817440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:02.818010 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:02.818230 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:02.818790 (+ 560us) server_negotiation.cc:299] Negotiation successful
0809 04:44:02.819260 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":372,"threads_started":1}
I20260809 04:44:02.823212 17623 raft_consensus.cc:348] T 00000000000000000000000000000000 P d486f07c9efd45de9596f2413265c847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d486f07c9efd45de9596f2413265c847" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 41207 } }
I20260809 04:44:02.823678 17623 raft_consensus.cc:374] T 00000000000000000000000000000000 P d486f07c9efd45de9596f2413265c847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:02.823796 17623 raft_consensus.cc:729] T 00000000000000000000000000000000 P d486f07c9efd45de9596f2413265c847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d486f07c9efd45de9596f2413265c847, State: Initialized, Role: FOLLOWER
I20260809 04:44:02.824414 17623 consensus_queue.cc:260] T 00000000000000000000000000000000 P d486f07c9efd45de9596f2413265c847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d486f07c9efd45de9596f2413265c847" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 41207 } }
I20260809 04:44:02.824674 17623 raft_consensus.cc:388] T 00000000000000000000000000000000 P d486f07c9efd45de9596f2413265c847 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:02.824817 17623 raft_consensus.cc:482] T 00000000000000000000000000000000 P d486f07c9efd45de9596f2413265c847 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:02.825008 17623 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d486f07c9efd45de9596f2413265c847 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:02.827354 17623 raft_consensus.cc:504] T 00000000000000000000000000000000 P d486f07c9efd45de9596f2413265c847 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d486f07c9efd45de9596f2413265c847" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 41207 } }
I20260809 04:44:02.827843 17623 leader_election.cc:302] T 00000000000000000000000000000000 P d486f07c9efd45de9596f2413265c847 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d486f07c9efd45de9596f2413265c847; no voters:
I20260809 04:44:02.828449 17623 leader_election.cc:288] T 00000000000000000000000000000000 P d486f07c9efd45de9596f2413265c847 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:02.828576 17628 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d486f07c9efd45de9596f2413265c847 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:02.829357 17628 raft_consensus.cc:686] T 00000000000000000000000000000000 P d486f07c9efd45de9596f2413265c847 [term 1 LEADER]: Becoming Leader. State: Replica: d486f07c9efd45de9596f2413265c847, State: Running, Role: LEADER
I20260809 04:44:02.829990 17628 consensus_queue.cc:237] T 00000000000000000000000000000000 P d486f07c9efd45de9596f2413265c847 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d486f07c9efd45de9596f2413265c847" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 41207 } }
I20260809 04:44:02.830775 17623 sys_catalog.cc:564] T 00000000000000000000000000000000 P d486f07c9efd45de9596f2413265c847 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:02.832875 17631 sys_catalog.cc:455] T 00000000000000000000000000000000 P d486f07c9efd45de9596f2413265c847 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d486f07c9efd45de9596f2413265c847. Latest consensus state: current_term: 1 leader_uuid: "d486f07c9efd45de9596f2413265c847" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d486f07c9efd45de9596f2413265c847" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 41207 } } }
I20260809 04:44:02.833029 17629 sys_catalog.cc:455] T 00000000000000000000000000000000 P d486f07c9efd45de9596f2413265c847 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d486f07c9efd45de9596f2413265c847" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d486f07c9efd45de9596f2413265c847" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 41207 } } }
I20260809 04:44:02.833529 17631 sys_catalog.cc:458] T 00000000000000000000000000000000 P d486f07c9efd45de9596f2413265c847 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:02.833524 17629 sys_catalog.cc:458] T 00000000000000000000000000000000 P d486f07c9efd45de9596f2413265c847 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:02.836992 17634 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:02.843592 17634 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:02.853121 17634 catalog_manager.cc:1347] Generated new cluster ID: 25e77437d2b244cb94b2613371c9ac73
I20260809 04:44:02.853258 17634 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:02.887910 17634 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:02.889518 17634 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:02.903287 17634 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d486f07c9efd45de9596f2413265c847: Generated new TSK 0
I20260809 04:44:02.904310 17634 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
I20260809 04:44:02.963923 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:0
--local_ip_for_outbound_sockets=127.12.234.1
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:41207
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:40295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:03.299240 17650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:03.299525 17650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:03.299624 17650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:03.306833 17650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:03.306939 17650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:03.307051 17650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:44:03.317306 17650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:40295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:41207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:03.318619 17650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:03.319957 17650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:03.334739 17655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:03.334802 17658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:03.334827 17656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:03.335816 17650 server_base.cc:1034] running on GCE node
I20260809 04:44:03.338135 17650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:03.339401 17650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:03.340651 17650 hybrid_clock.cc:648] HybridClock initialized: now 1786250643340608 us; error 33 us; skew 500 ppm
I20260809 04:44:03.341162 17650 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:03 dist-test-slave-1jjb krb5kdc[17539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250643, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:03.345515 17650 init.cc:377] Logged in from keytab as kudu/127.12.234.1@KRBTEST.COM (short username kudu)
I20260809 04:44:03.347450 17650 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:03.347640 17650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:03.348141 17650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:03.352422 17650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-0/data/instance:
uuid: "14f188dee16a4223b2fd21c506166205"
format_stamp: "Formatted at 2026-08-09 04:44:03 on dist-test-slave-1jjb"
server_key: "ecbf4d6c1712f1eea36d957a580af910"
server_key_iv: "fa868bdea146873807ee19b78b2476e0"
server_key_version: "encryptionkey@0"
I20260809 04:44:03.353516 17650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance:
uuid: "14f188dee16a4223b2fd21c506166205"
format_stamp: "Formatted at 2026-08-09 04:44:03 on dist-test-slave-1jjb"
server_key: "ecbf4d6c1712f1eea36d957a580af910"
server_key_iv: "fa868bdea146873807ee19b78b2476e0"
server_key_version: "encryptionkey@0"
I20260809 04:44:03.360328 17650 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260809 04:44:03.364377 17664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:03.366053 17650 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:44:03.366289 17650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "14f188dee16a4223b2fd21c506166205"
format_stamp: "Formatted at 2026-08-09 04:44:03 on dist-test-slave-1jjb"
server_key: "ecbf4d6c1712f1eea36d957a580af910"
server_key_iv: "fa868bdea146873807ee19b78b2476e0"
server_key_version: "encryptionkey@0"
I20260809 04:44:03.366573 17650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:03.394016 17650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:03.398604 17650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:03.398988 17650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:03.400007 17650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:03.401486 17650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:03.401613 17650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:03.401749 17650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:03.401839 17650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:03.436275 17650 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:32963
I20260809 04:44:03.436359 17783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:32963 every 8 connection(s)
I20260809 04:44:03.437300 17650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:44:03.446105 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 17650
I20260809 04:44:03.446319 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance
I20260809 04:44:03.447083 13224 external_mini_cluster.cc:1428] Setting key c69567463d38dbc48947bf507220d33a
Aug 09 04:44:03 dist-test-slave-1jjb krb5kdc[17539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250643, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:44:03.464607 17786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:03.440587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:34029 (local address 127.12.234.62:41207)
0809 04:44:03.441090 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:03.441097 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:03.442012 (+ 915us) server_negotiation.cc:407] Connection header received
0809 04:44:03.443815 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:03.443820 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:03.443896 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:03.443993 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:03.445557 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:03.446370 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:03.448255 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:03.448839 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:03.448969 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:03.457847 (+ 8878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:03.457854 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:03.457857 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:03.457899 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:03.460891 (+ 2992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:03.461993 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:03.461997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:03.462000 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:03.462121 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:03.462633 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:03.462636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:03.462638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:03.462917 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:03.463029 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:03.464125 (+ 1096us) server_negotiation.cc:299] Negotiation successful
0809 04:44:03.464324 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":226,"threads_started":1}
I20260809 04:44:03.466279 17787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:03.441127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:41207 (local address 127.12.234.1:34029)
0809 04:44:03.441849 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:03.441894 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:03.443555 (+ 1661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:03.444269 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:03.444278 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:03.444800 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:03.445372 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:03.445389 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:03.446589 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:03.446593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:03.448062 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:03.448076 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:03.449128 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:03.449139 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:03.449351 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:03.454593 (+ 5242us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:03.454636 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:03.457504 (+ 2868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:03.461072 (+ 3568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:03.461086 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:03.461109 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:03.461847 (+ 738us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:03.462236 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:03.462244 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:03.462251 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:03.462504 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:03.463057 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:03.463072 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:03.463631 (+ 559us) client_negotiation.cc:769] Sending connection context
0809 04:44:03.463974 (+ 343us) client_negotiation.cc:240] Negotiation successful
0809 04:44:03.464626 (+ 652us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":307,"threads_started":1}
I20260809 04:44:03.468063 17784 heartbeater.cc:347] Connected to a master server at 127.12.234.62:41207
I20260809 04:44:03.468452 17784 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:03.469162 17784 heartbeater.cc:511] Master 127.12.234.62:41207 requested a full tablet report, sending...
I20260809 04:44:03.471854 17584 ts_manager.cc:194] Registered new tserver with Master: 14f188dee16a4223b2fd21c506166205 (127.12.234.1:32963)
WARNING: no policy specified for kudu/127.12.234.2@KRBTEST.COM; defaulting to no policy
I20260809 04:44:03.474514 17584 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.1@KRBTEST.COM'} at 127.12.234.1:34029
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
I20260809 04:44:03.505251 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:0
--local_ip_for_outbound_sockets=127.12.234.2
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:41207
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:40295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:03.847101 17790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:03.847502 17790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:03.847692 17790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:03.855155 17790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:03.855306 17790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:03.855512 17790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:44:03.865289 17790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:40295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:41207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:03.866720 17790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:03.868016 17790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:03.881047 17795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:03.881031 17798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:03.881098 17796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:03.881881 17790 server_base.cc:1034] running on GCE node
I20260809 04:44:03.882395 17790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:03.883257 17790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:03.884456 17790 hybrid_clock.cc:648] HybridClock initialized: now 1786250643884402 us; error 52 us; skew 500 ppm
I20260809 04:44:03.884728 17790 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:03 dist-test-slave-1jjb krb5kdc[17539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250643, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:03.887686 17790 init.cc:377] Logged in from keytab as kudu/127.12.234.2@KRBTEST.COM (short username kudu)
I20260809 04:44:03.888780 17790 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:03.888865 17790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:03.889149 17790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:03.891901 17790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-1/data/instance:
uuid: "56edc2c22f004f4fb9f5310786fc1a00"
format_stamp: "Formatted at 2026-08-09 04:44:03 on dist-test-slave-1jjb"
server_key: "508eb84466cc9be3ef4ecf92a163a8d9"
server_key_iv: "ccf8b1f0a8f79c1694bdcc3ef468a3fa"
server_key_version: "encryptionkey@0"
I20260809 04:44:03.892714 17790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance:
uuid: "56edc2c22f004f4fb9f5310786fc1a00"
format_stamp: "Formatted at 2026-08-09 04:44:03 on dist-test-slave-1jjb"
server_key: "508eb84466cc9be3ef4ecf92a163a8d9"
server_key_iv: "ccf8b1f0a8f79c1694bdcc3ef468a3fa"
server_key_version: "encryptionkey@0"
I20260809 04:44:03.897562 17790 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 04:44:03.900638 17805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:03.901890 17790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:44:03.902091 17790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "56edc2c22f004f4fb9f5310786fc1a00"
format_stamp: "Formatted at 2026-08-09 04:44:03 on dist-test-slave-1jjb"
server_key: "508eb84466cc9be3ef4ecf92a163a8d9"
server_key_iv: "ccf8b1f0a8f79c1694bdcc3ef468a3fa"
server_key_version: "encryptionkey@0"
I20260809 04:44:03.902303 17790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:03.915259 17790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:03.919029 17790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:03.919286 17790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:03.920259 17790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:03.921618 17790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:03.921739 17790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:03.921868 17790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:03.921957 17790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:03.955569 17790 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:38673
I20260809 04:44:03.955654 17922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:38673 every 8 connection(s)
I20260809 04:44:03.956470 17790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:44:03.958560 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 17790
I20260809 04:44:03.958750 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance
I20260809 04:44:03.959188 13224 external_mini_cluster.cc:1428] Setting key 7aa4926e4ce6b1c9c564e5b88b4982f3
Aug 09 04:44:03 dist-test-slave-1jjb krb5kdc[17539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250643, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:44:03.976753 17786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:03.959940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:52251 (local address 127.12.234.62:41207)
0809 04:44:03.960361 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:03.960367 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:03.960386 (+ 19us) server_negotiation.cc:407] Connection header received
0809 04:44:03.962270 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:03.962280 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:03.962379 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:03.962490 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:03.963865 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:03.964986 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:03.966505 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:03.967225 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:03.967368 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:03.971151 (+ 3783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:03.971163 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:03.971185 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:03.971229 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:03.973750 (+ 2521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:03.974476 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:03.974483 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:03.974487 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:03.974614 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:03.975025 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:03.975030 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:03.975033 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:03.975318 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:03.975477 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:03.976165 (+ 688us) server_negotiation.cc:299] Negotiation successful
0809 04:44:03.976403 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308}
I20260809 04:44:03.977749 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:03.959376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:41207 (local address 127.12.234.2:52251)
0809 04:44:03.960184 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:03.960237 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:03.962003 (+ 1766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:03.962674 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:03.962681 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:03.963167 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:03.963684 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:03.963699 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:03.965150 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:03.965153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:03.966348 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:03.966359 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:03.967464 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:03.967472 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:03.967680 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:03.968369 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:03.968394 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:03.970911 (+ 2517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:03.973898 (+ 2987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:03.973907 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:03.973922 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:03.974357 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:03.974725 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:03.974730 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:03.974734 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:03.974913 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:03.975447 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:03.975455 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:03.975785 (+ 330us) client_negotiation.cc:769] Sending connection context
0809 04:44:03.976052 (+ 267us) client_negotiation.cc:240] Negotiation successful
0809 04:44:03.976614 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":324,"threads_started":1}
I20260809 04:44:03.979507 17923 heartbeater.cc:347] Connected to a master server at 127.12.234.62:41207
I20260809 04:44:03.979892 17923 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:03.980638 17923 heartbeater.cc:511] Master 127.12.234.62:41207 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.12.234.3@KRBTEST.COM; defaulting to no policy
I20260809 04:44:03.982218 17584 ts_manager.cc:194] Registered new tserver with Master: 56edc2c22f004f4fb9f5310786fc1a00 (127.12.234.2:38673)
I20260809 04:44:03.983065 17584 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.2@KRBTEST.COM'} at 127.12.234.2:52251
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
I20260809 04:44:04.011281 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:0
--local_ip_for_outbound_sockets=127.12.234.3
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:41207
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:40295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:04.340493 17928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:04.340847 17928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:04.340952 17928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:04.348239 17928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:04.348335 17928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:04.348453 17928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:44:04.357911 17928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:40295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:41207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:04.359093 17928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:04.360292 17928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:04.373139 17937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:04.373168 17935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:04.373374 17934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:04.374013 17928 server_base.cc:1034] running on GCE node
I20260809 04:44:04.374632 17928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:04.375497 17928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:04.376699 17928 hybrid_clock.cc:648] HybridClock initialized: now 1786250644376656 us; error 40 us; skew 500 ppm
I20260809 04:44:04.377010 17928 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:04 dist-test-slave-1jjb krb5kdc[17539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250644, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:04.380257 17928 init.cc:377] Logged in from keytab as kudu/127.12.234.3@KRBTEST.COM (short username kudu)
I20260809 04:44:04.381460 17928 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:04.381598 17928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:04.381915 17928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:04.384847 17928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-2/data/instance:
uuid: "b4d2215b9e674777ad1f904eda053586"
format_stamp: "Formatted at 2026-08-09 04:44:04 on dist-test-slave-1jjb"
server_key: "54f2fdacec0ddfa14801d301eb9b732b"
server_key_iv: "3db85411f6ec6b693f726b520cc76d8b"
server_key_version: "encryptionkey@0"
I20260809 04:44:04.385691 17928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance:
uuid: "b4d2215b9e674777ad1f904eda053586"
format_stamp: "Formatted at 2026-08-09 04:44:04 on dist-test-slave-1jjb"
server_key: "54f2fdacec0ddfa14801d301eb9b732b"
server_key_iv: "3db85411f6ec6b693f726b520cc76d8b"
server_key_version: "encryptionkey@0"
I20260809 04:44:04.390537 17928 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 04:44:04.394112 17943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:04.395357 17928 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:44:04.395535 17928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "b4d2215b9e674777ad1f904eda053586"
format_stamp: "Formatted at 2026-08-09 04:44:04 on dist-test-slave-1jjb"
server_key: "54f2fdacec0ddfa14801d301eb9b732b"
server_key_iv: "3db85411f6ec6b693f726b520cc76d8b"
server_key_version: "encryptionkey@0"
I20260809 04:44:04.395766 17928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:04.413084 17928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:04.417906 17928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:04.418216 17928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:04.419235 17928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:04.420725 17928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:04.420878 17928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:04.421015 17928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:04.421120 17928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:04.458081 17928 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:44199
I20260809 04:44:04.458097 18061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:44199 every 8 connection(s)
I20260809 04:44:04.458992 17928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:44:04.463035 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 17928
I20260809 04:44:04.463243 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance
I20260809 04:44:04.463694 13224 external_mini_cluster.cc:1428] Setting key 7ed8d786c627f58b622bf92bc1b15901
Aug 09 04:44:04 dist-test-slave-1jjb krb5kdc[17539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250644, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:44:04.478780 17784 heartbeater.cc:503] Master 127.12.234.62:41207 was elected leader, sending a full tablet report...
I20260809 04:44:04.480793 17786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:04.461706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:44459 (local address 127.12.234.62:41207)
0809 04:44:04.461889 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:04.461893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:04.463111 (+ 1218us) server_negotiation.cc:407] Connection header received
0809 04:44:04.465528 (+ 2417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:04.465536 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:04.465669 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:04.465793 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:04.467388 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.468245 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.469899 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.470441 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.470647 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:04.474335 (+ 3688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:04.474342 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:04.474349 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:04.474389 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:04.477232 (+ 2843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:04.477977 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:04.477984 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:04.477987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:04.478118 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:04.478951 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:04.478956 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:04.478959 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:04.479346 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:04.479509 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:04.480188 (+ 679us) server_negotiation.cc:299] Negotiation successful
0809 04:44:04.480421 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 04:44:04.481666 18064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:04.462057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:41207 (local address 127.12.234.3:44459)
0809 04:44:04.462928 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:04.462969 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:04.465226 (+ 2257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:04.466034 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:04.466043 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:04.466595 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:04.467182 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:04.467206 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.468408 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.468419 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:04.469730 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:04.469744 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.470759 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.470768 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:04.471030 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:04.471753 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:04.471778 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:04.474122 (+ 2344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:04.477398 (+ 3276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:04.477407 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:04.477422 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:04.477855 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:04.478578 (+ 723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:04.478583 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:04.478586 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:04.478810 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:04.479491 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:04.479497 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:04.479789 (+ 292us) client_negotiation.cc:769] Sending connection context
0809 04:44:04.480066 (+ 277us) client_negotiation.cc:240] Negotiation successful
0809 04:44:04.480595 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":322,"threads_started":1}
I20260809 04:44:04.483016 18062 heartbeater.cc:347] Connected to a master server at 127.12.234.62:41207
I20260809 04:44:04.483328 18062 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:04.484000 18062 heartbeater.cc:511] Master 127.12.234.62:41207 requested a full tablet report, sending...
I20260809 04:44:04.485328 17584 ts_manager.cc:194] Registered new tserver with Master: b4d2215b9e674777ad1f904eda053586 (127.12.234.3:44199)
I20260809 04:44:04.486078 17584 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.3@KRBTEST.COM'} at 127.12.234.3:44459
I20260809 04:44:04.490770 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:44:04.507550 17786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:04.495408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50564 (local address 127.12.234.62:41207)
0809 04:44:04.495590 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:04.495595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:04.496162 (+ 567us) server_negotiation.cc:407] Connection header received
0809 04:44:04.496389 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:04.496394 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:04.496452 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:04.496556 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:04.497974 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.498733 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.500030 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.500534 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.500669 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:04.502757 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:04.502766 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:04.502775 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:04.502803 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:04.504998 (+ 2195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:04.505627 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:04.505633 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:04.505637 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:04.505765 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:04.506155 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:04.506160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:04.506165 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:04.506479 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:04.506744 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:04.507066 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 04:44:04.507267 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 04:44:04.511548 17584 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:44:04.518226 17584 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:44:04.547807 18074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:04.530878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:44199 (local address 127.0.0.1:38396)
0809 04:44:04.531647 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:04.531685 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:04.531858 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:04.539724 (+ 7866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:04.539732 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:04.539761 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:04.540177 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:04.540195 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.542532 (+ 2337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.542538 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:04.544939 (+ 2401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:04.544958 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.546743 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.546750 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:04.546934 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:04.546985 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 04:44:04.547109 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 04:44:04.547331 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":32,"thread_start_us":764,"threads_started":1}
I20260809 04:44:04.548451 18075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:04.531271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:32963 (local address 127.0.0.1:52942)
0809 04:44:04.538363 (+ 7092us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:04.538387 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:04.538522 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:04.541256 (+ 2734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:04.541261 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:04.541289 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:04.541704 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:04.541717 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.543796 (+ 2079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.543804 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:04.545768 (+ 1964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:04.545785 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.547603 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.547615 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:04.547795 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:04.547829 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 04:44:04.547930 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 04:44:04.548139 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6930,"thread_start_us":6785,"threads_started":1}
I20260809 04:44:04.548761 18076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:04.531374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38396 (local address 127.12.234.3:44199)
0809 04:44:04.539210 (+ 7836us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:04.539217 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:04.539244 (+ 27us) server_negotiation.cc:407] Connection header received
0809 04:44:04.539321 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:04.539326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:04.539511 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:04.539718 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:04.540447 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.542366 (+ 1919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.545104 (+ 2738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.546233 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.546360 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:04.546611 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:04.548281 (+ 1670us) server_negotiation.cc:299] Negotiation successful
0809 04:44:04.548424 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7674,"thread_start_us":7487,"threads_started":1}
I20260809 04:44:04.549173 18077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:04.531831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52942 (local address 127.12.234.1:32963)
0809 04:44:04.540480 (+ 8649us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:04.540488 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:04.540514 (+ 26us) server_negotiation.cc:407] Connection header received
0809 04:44:04.540761 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:04.540772 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:04.540999 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:04.541926 (+ 927us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:04.542148 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.543643 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.545949 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.547399 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.547536 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:04.547747 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:04.548671 (+ 924us) server_negotiation.cc:299] Negotiation successful
0809 04:44:04.548825 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8447,"thread_start_us":8267,"threads_started":1}
I20260809 04:44:04.551438 18081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:04.542772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:38673 (local address 127.0.0.1:33366)
0809 04:44:04.543382 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:04.543402 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:04.543550 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:04.544763 (+ 1213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:04.544767 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:04.544793 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:04.545284 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:04.545297 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.547009 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.547014 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:04.549116 (+ 2102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:04.549131 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.550689 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.550696 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:04.550854 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:04.550900 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 04:44:04.551004 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 04:44:04.551122 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":431,"threads_started":1}
I20260809 04:44:04.551508 18078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:04.538755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33366 (local address 127.12.234.2:38673)
0809 04:44:04.544239 (+ 5484us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:04.544247 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:04.544273 (+ 26us) server_negotiation.cc:407] Connection header received
0809 04:44:04.544374 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:04.544381 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:04.544582 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:04.544749 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:04.545487 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.546859 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.549311 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.550491 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.550649 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:04.550823 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:04.551055 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 04:44:04.551208 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5295,"thread_start_us":5197,"threads_started":1}
I20260809 04:44:04.552809 17992 tablet_service.cc:1467] Processing CreateTablet for tablet 28d3e7ade5064d449360b8ae7aac50c6 (DEFAULT_TABLE table=test-table [id=6db0d36c3e3d4d728452ac762fd25038]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:04.552807 17715 tablet_service.cc:1467] Processing CreateTablet for tablet 28d3e7ade5064d449360b8ae7aac50c6 (DEFAULT_TABLE table=test-table [id=6db0d36c3e3d4d728452ac762fd25038]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:04.554211 17715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28d3e7ade5064d449360b8ae7aac50c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:04.554231 17992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28d3e7ade5064d449360b8ae7aac50c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:04.558768 17854 tablet_service.cc:1467] Processing CreateTablet for tablet 28d3e7ade5064d449360b8ae7aac50c6 (DEFAULT_TABLE table=test-table [id=6db0d36c3e3d4d728452ac762fd25038]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:04.560209 17854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28d3e7ade5064d449360b8ae7aac50c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:04.572325 18084 tablet_bootstrap.cc:492] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00: Bootstrap starting.
I20260809 04:44:04.574292 18082 tablet_bootstrap.cc:492] T 28d3e7ade5064d449360b8ae7aac50c6 P 14f188dee16a4223b2fd21c506166205: Bootstrap starting.
I20260809 04:44:04.576126 18084 tablet_bootstrap.cc:654] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:04.576828 18083 tablet_bootstrap.cc:492] T 28d3e7ade5064d449360b8ae7aac50c6 P b4d2215b9e674777ad1f904eda053586: Bootstrap starting.
I20260809 04:44:04.577481 18084 log.cc:824] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:04.577819 18082 tablet_bootstrap.cc:654] T 28d3e7ade5064d449360b8ae7aac50c6 P 14f188dee16a4223b2fd21c506166205: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:04.579600 18082 log.cc:824] T 28d3e7ade5064d449360b8ae7aac50c6 P 14f188dee16a4223b2fd21c506166205: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:04.580716 18083 tablet_bootstrap.cc:654] T 28d3e7ade5064d449360b8ae7aac50c6 P b4d2215b9e674777ad1f904eda053586: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:04.581089 18084 tablet_bootstrap.cc:492] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00: No bootstrap required, opened a new log
I20260809 04:44:04.581388 18084 ts_tablet_manager.cc:1397] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 04:44:04.581807 18083 log.cc:824] T 28d3e7ade5064d449360b8ae7aac50c6 P b4d2215b9e674777ad1f904eda053586: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:04.582738 18082 tablet_bootstrap.cc:492] T 28d3e7ade5064d449360b8ae7aac50c6 P 14f188dee16a4223b2fd21c506166205: No bootstrap required, opened a new log
I20260809 04:44:04.583009 18082 ts_tablet_manager.cc:1397] T 28d3e7ade5064d449360b8ae7aac50c6 P 14f188dee16a4223b2fd21c506166205: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.002s
I20260809 04:44:04.584398 18083 tablet_bootstrap.cc:492] T 28d3e7ade5064d449360b8ae7aac50c6 P b4d2215b9e674777ad1f904eda053586: No bootstrap required, opened a new log
I20260809 04:44:04.584662 18083 ts_tablet_manager.cc:1397] T 28d3e7ade5064d449360b8ae7aac50c6 P b4d2215b9e674777ad1f904eda053586: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 04:44:04.591583 18083 raft_consensus.cc:348] T 28d3e7ade5064d449360b8ae7aac50c6 P b4d2215b9e674777ad1f904eda053586 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4d2215b9e674777ad1f904eda053586" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 44199 } } peers { permanent_uuid: "14f188dee16a4223b2fd21c506166205" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32963 } } peers { permanent_uuid: "56edc2c22f004f4fb9f5310786fc1a00" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38673 } }
I20260809 04:44:04.592037 18083 raft_consensus.cc:374] T 28d3e7ade5064d449360b8ae7aac50c6 P b4d2215b9e674777ad1f904eda053586 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:04.592181 18083 raft_consensus.cc:729] T 28d3e7ade5064d449360b8ae7aac50c6 P b4d2215b9e674777ad1f904eda053586 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4d2215b9e674777ad1f904eda053586, State: Initialized, Role: FOLLOWER
I20260809 04:44:04.592099 18084 raft_consensus.cc:348] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4d2215b9e674777ad1f904eda053586" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 44199 } } peers { permanent_uuid: "14f188dee16a4223b2fd21c506166205" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32963 } } peers { permanent_uuid: "56edc2c22f004f4fb9f5310786fc1a00" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38673 } }
I20260809 04:44:04.592517 18084 raft_consensus.cc:374] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:04.592602 18084 raft_consensus.cc:729] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56edc2c22f004f4fb9f5310786fc1a00, State: Initialized, Role: FOLLOWER
I20260809 04:44:04.592824 18083 consensus_queue.cc:260] T 28d3e7ade5064d449360b8ae7aac50c6 P b4d2215b9e674777ad1f904eda053586 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4d2215b9e674777ad1f904eda053586" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 44199 } } peers { permanent_uuid: "14f188dee16a4223b2fd21c506166205" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32963 } } peers { permanent_uuid: "56edc2c22f004f4fb9f5310786fc1a00" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38673 } }
I20260809 04:44:04.593328 18084 consensus_queue.cc:260] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4d2215b9e674777ad1f904eda053586" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 44199 } } peers { permanent_uuid: "14f188dee16a4223b2fd21c506166205" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32963 } } peers { permanent_uuid: "56edc2c22f004f4fb9f5310786fc1a00" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38673 } }
I20260809 04:44:04.594378 18062 heartbeater.cc:503] Master 127.12.234.62:41207 was elected leader, sending a full tablet report...
I20260809 04:44:04.594813 18083 ts_tablet_manager.cc:1428] T 28d3e7ade5064d449360b8ae7aac50c6 P b4d2215b9e674777ad1f904eda053586: Time spent starting tablet: real 0.010s user 0.005s sys 0.004s
I20260809 04:44:04.593727 18082 raft_consensus.cc:348] T 28d3e7ade5064d449360b8ae7aac50c6 P 14f188dee16a4223b2fd21c506166205 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4d2215b9e674777ad1f904eda053586" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 44199 } } peers { permanent_uuid: "14f188dee16a4223b2fd21c506166205" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32963 } } peers { permanent_uuid: "56edc2c22f004f4fb9f5310786fc1a00" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38673 } }
I20260809 04:44:04.595094 18082 raft_consensus.cc:374] T 28d3e7ade5064d449360b8ae7aac50c6 P 14f188dee16a4223b2fd21c506166205 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:04.595220 18082 raft_consensus.cc:729] T 28d3e7ade5064d449360b8ae7aac50c6 P 14f188dee16a4223b2fd21c506166205 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14f188dee16a4223b2fd21c506166205, State: Initialized, Role: FOLLOWER
I20260809 04:44:04.595788 18082 consensus_queue.cc:260] T 28d3e7ade5064d449360b8ae7aac50c6 P 14f188dee16a4223b2fd21c506166205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4d2215b9e674777ad1f904eda053586" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 44199 } } peers { permanent_uuid: "14f188dee16a4223b2fd21c506166205" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32963 } } peers { permanent_uuid: "56edc2c22f004f4fb9f5310786fc1a00" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38673 } }
I20260809 04:44:04.596614 17923 heartbeater.cc:503] Master 127.12.234.62:41207 was elected leader, sending a full tablet report...
I20260809 04:44:04.597291 18084 ts_tablet_manager.cc:1428] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00: Time spent starting tablet: real 0.016s user 0.013s sys 0.003s
I20260809 04:44:04.599011 18082 ts_tablet_manager.cc:1428] T 28d3e7ade5064d449360b8ae7aac50c6 P 14f188dee16a4223b2fd21c506166205: Time spent starting tablet: real 0.016s user 0.014s sys 0.001s
W20260809 04:44:04.690515 17785 tablet.cc:2369] T 28d3e7ade5064d449360b8ae7aac50c6 P 14f188dee16a4223b2fd21c506166205: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:44:04.695806 18089 raft_consensus.cc:482] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:44:04.696202 18089 raft_consensus.cc:504] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4d2215b9e674777ad1f904eda053586" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 44199 } } peers { permanent_uuid: "14f188dee16a4223b2fd21c506166205" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32963 } } peers { permanent_uuid: "56edc2c22f004f4fb9f5310786fc1a00" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38673 } }
I20260809 04:44:04.697978 18089 leader_election.cc:288] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b4d2215b9e674777ad1f904eda053586 (127.12.234.3:44199), 14f188dee16a4223b2fd21c506166205 (127.12.234.1:32963)
I20260809 04:44:04.705637 18077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:04.698452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:58477 (local address 127.12.234.1:32963)
0809 04:44:04.698683 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:04.698690 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:04.699871 (+ 1181us) server_negotiation.cc:407] Connection header received
0809 04:44:04.700164 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:04.700172 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:04.700252 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:04.700354 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:04.701062 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.701776 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.703710 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.704576 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.704732 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:04.704826 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:04.705194 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 04:44:04.705323 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 04:44:04.705674 18076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:04.698483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:54369 (local address 127.12.234.3:44199)
0809 04:44:04.698696 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:04.698705 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:04.699785 (+ 1080us) server_negotiation.cc:407] Connection header received
0809 04:44:04.699987 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:04.699998 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:04.700062 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:04.700202 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:04.700955 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.701745 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.703757 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.704607 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.704805 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:04.704876 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:04.705241 (+ 365us) server_negotiation.cc:299] Negotiation successful
0809 04:44:04.705375 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 04:44:04.705861 18092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:04.698540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:32963 (local address 127.12.234.2:58477)
0809 04:44:04.699773 (+ 1233us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:04.699793 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:04.699994 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:04.700406 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:04.700411 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:04.700440 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:04.700889 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:04.700903 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.702043 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.702047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:04.703546 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:04.703563 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.704818 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.704826 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:04.705023 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:04.705055 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 04:44:04.705139 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 04:44:04.705340 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1089,"mutex_wait_us":22,"thread_start_us":656,"threads_started":1}
I20260809 04:44:04.706225 18091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:04.698472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:44199 (local address 127.12.234.2:54369)
0809 04:44:04.699672 (+ 1200us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:04.699690 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:04.699829 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:04.700223 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:04.700228 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:04.700251 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:04.700774 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:04.700799 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.701981 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.701987 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:04.703571 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:04.703583 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.704857 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.704864 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:04.705052 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:04.705075 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 04:44:04.705193 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 04:44:04.705323 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1046,"mutex_wait_us":69,"thread_start_us":594,"threads_started":1}
I20260809 04:44:04.707008 17736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d3e7ade5064d449360b8ae7aac50c6" candidate_uuid: "56edc2c22f004f4fb9f5310786fc1a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14f188dee16a4223b2fd21c506166205" is_pre_election: true
I20260809 04:44:04.707194 18014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d3e7ade5064d449360b8ae7aac50c6" candidate_uuid: "56edc2c22f004f4fb9f5310786fc1a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4d2215b9e674777ad1f904eda053586" is_pre_election: true
I20260809 04:44:04.707569 17736 raft_consensus.cc:2454] T 28d3e7ade5064d449360b8ae7aac50c6 P 14f188dee16a4223b2fd21c506166205 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56edc2c22f004f4fb9f5310786fc1a00 in term 0.
I20260809 04:44:04.707752 18014 raft_consensus.cc:2454] T 28d3e7ade5064d449360b8ae7aac50c6 P b4d2215b9e674777ad1f904eda053586 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56edc2c22f004f4fb9f5310786fc1a00 in term 0.
I20260809 04:44:04.708293 17809 leader_election.cc:302] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14f188dee16a4223b2fd21c506166205, 56edc2c22f004f4fb9f5310786fc1a00; no voters:
W20260809 04:44:04.708405 17924 tablet.cc:2369] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:44:04.708745 18089 raft_consensus.cc:2781] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:44:04.708909 18089 raft_consensus.cc:482] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:44:04.709024 18089 raft_consensus.cc:3037] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00 [term 0 FOLLOWER]: Advancing to term 1
W20260809 04:44:04.710539 18063 tablet.cc:2369] T 28d3e7ade5064d449360b8ae7aac50c6 P b4d2215b9e674777ad1f904eda053586: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:44:04.711455 18089 raft_consensus.cc:504] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4d2215b9e674777ad1f904eda053586" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 44199 } } peers { permanent_uuid: "14f188dee16a4223b2fd21c506166205" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32963 } } peers { permanent_uuid: "56edc2c22f004f4fb9f5310786fc1a00" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38673 } }
I20260809 04:44:04.712137 18089 leader_election.cc:288] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00 [CANDIDATE]: Term 1 election: Requested vote from peers b4d2215b9e674777ad1f904eda053586 (127.12.234.3:44199), 14f188dee16a4223b2fd21c506166205 (127.12.234.1:32963)
I20260809 04:44:04.712585 18014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d3e7ade5064d449360b8ae7aac50c6" candidate_uuid: "56edc2c22f004f4fb9f5310786fc1a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b4d2215b9e674777ad1f904eda053586"
I20260809 04:44:04.712637 17736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28d3e7ade5064d449360b8ae7aac50c6" candidate_uuid: "56edc2c22f004f4fb9f5310786fc1a00" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14f188dee16a4223b2fd21c506166205"
I20260809 04:44:04.712805 18014 raft_consensus.cc:3037] T 28d3e7ade5064d449360b8ae7aac50c6 P b4d2215b9e674777ad1f904eda053586 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:04.712911 17736 raft_consensus.cc:3037] T 28d3e7ade5064d449360b8ae7aac50c6 P 14f188dee16a4223b2fd21c506166205 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:04.715104 18014 raft_consensus.cc:2454] T 28d3e7ade5064d449360b8ae7aac50c6 P b4d2215b9e674777ad1f904eda053586 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56edc2c22f004f4fb9f5310786fc1a00 in term 1.
I20260809 04:44:04.715188 17736 raft_consensus.cc:2454] T 28d3e7ade5064d449360b8ae7aac50c6 P 14f188dee16a4223b2fd21c506166205 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56edc2c22f004f4fb9f5310786fc1a00 in term 1.
I20260809 04:44:04.715648 17806 leader_election.cc:302] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56edc2c22f004f4fb9f5310786fc1a00, b4d2215b9e674777ad1f904eda053586; no voters:
I20260809 04:44:04.715972 18089 raft_consensus.cc:2781] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:04.716455 18089 raft_consensus.cc:686] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00 [term 1 LEADER]: Becoming Leader. State: Replica: 56edc2c22f004f4fb9f5310786fc1a00, State: Running, Role: LEADER
I20260809 04:44:04.716948 18089 consensus_queue.cc:237] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4d2215b9e674777ad1f904eda053586" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 44199 } } peers { permanent_uuid: "14f188dee16a4223b2fd21c506166205" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32963 } } peers { permanent_uuid: "56edc2c22f004f4fb9f5310786fc1a00" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38673 } }
I20260809 04:44:04.721894 17582 catalog_manager.cc:5495] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56edc2c22f004f4fb9f5310786fc1a00 (127.12.234.2). New cstate: current_term: 1 leader_uuid: "56edc2c22f004f4fb9f5310786fc1a00" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4d2215b9e674777ad1f904eda053586" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 44199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14f188dee16a4223b2fd21c506166205" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56edc2c22f004f4fb9f5310786fc1a00" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38673 } health_report { overall_health: HEALTHY } } }
I20260809 04:44:04.781742 18078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:04.776130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33380 (local address 127.12.234.2:38673)
0809 04:44:04.776355 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:04.776363 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:04.776465 (+ 102us) server_negotiation.cc:407] Connection header received
0809 04:44:04.776736 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:04.776741 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:04.776832 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:04.777004 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:44:04.777408 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.778112 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.779504 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.779915 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.780056 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:04.780589 (+ 533us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:44:04.781123 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:44:04.781311 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:04.781366 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 04:44:04.781449 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:44:04.791225 17736 raft_consensus.cc:1261] T 28d3e7ade5064d449360b8ae7aac50c6 P 14f188dee16a4223b2fd21c506166205 [term 1 FOLLOWER]: Refusing update from remote peer 56edc2c22f004f4fb9f5310786fc1a00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:04.791317 18014 raft_consensus.cc:1261] T 28d3e7ade5064d449360b8ae7aac50c6 P b4d2215b9e674777ad1f904eda053586 [term 1 FOLLOWER]: Refusing update from remote peer 56edc2c22f004f4fb9f5310786fc1a00: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:04.792377 18093 consensus_queue.cc:1035] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14f188dee16a4223b2fd21c506166205" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 32963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:04.792821 18089 consensus_queue.cc:1035] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4d2215b9e674777ad1f904eda053586" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 44199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:04.809358 18099 mvcc.cc:205] Tried to move back new op lower bound from 7316482641053351936 to 7316482640764862464. Current Snapshot: MvccSnapshot[applied={T|T < 7316482641053351936}]
I20260809 04:44:04.829260 17584 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:04.829597 17584 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:04.833458 17584 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d486f07c9efd45de9596f2413265c847: Sending DeleteTablet for 3 replicas of tablet 28d3e7ade5064d449360b8ae7aac50c6
I20260809 04:44:04.834782 17715 tablet_service.cc:1514] Processing DeleteTablet for tablet 28d3e7ade5064d449360b8ae7aac50c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:04 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:52942
I20260809 04:44:04.835268 17854 tablet_service.cc:1514] Processing DeleteTablet for tablet 28d3e7ade5064d449360b8ae7aac50c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:04 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:33366
I20260809 04:44:04.835584 17992 tablet_service.cc:1514] Processing DeleteTablet for tablet 28d3e7ade5064d449360b8ae7aac50c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:04 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:38396
I20260809 04:44:04.835951 18107 tablet_replica.cc:330] T 28d3e7ade5064d449360b8ae7aac50c6 P 14f188dee16a4223b2fd21c506166205: stopping tablet replica
I20260809 04:44:04.836601 18107 raft_consensus.cc:2229] T 28d3e7ade5064d449360b8ae7aac50c6 P 14f188dee16a4223b2fd21c506166205 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:44:04.837314 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 17650
I20260809 04:44:04.837517 18107 raft_consensus.cc:2258] T 28d3e7ade5064d449360b8ae7aac50c6 P 14f188dee16a4223b2fd21c506166205 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:04.840296 18107 ts_tablet_manager.cc:1905] T 28d3e7ade5064d449360b8ae7aac50c6 P 14f188dee16a4223b2fd21c506166205: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:04.848183 18108 tablet_replica.cc:330] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00: stopping tablet replica
I20260809 04:44:04.848737 18108 raft_consensus.cc:2229] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:44:04.849520 18108 raft_consensus.cc:2258] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:04.850607 18107 ts_tablet_manager.cc:1918] T 28d3e7ade5064d449360b8ae7aac50c6 P 14f188dee16a4223b2fd21c506166205: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:04.850931 18107 log.cc:1196] T 28d3e7ade5064d449360b8ae7aac50c6 P 14f188dee16a4223b2fd21c506166205: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-0/wal/wals/28d3e7ade5064d449360b8ae7aac50c6
I20260809 04:44:04.851660 18107 ts_tablet_manager.cc:1939] T 28d3e7ade5064d449360b8ae7aac50c6 P 14f188dee16a4223b2fd21c506166205: Deleting consensus metadata
I20260809 04:44:04.852941 18108 ts_tablet_manager.cc:1905] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:04.853911 17571 catalog_manager.cc:4841] TS 14f188dee16a4223b2fd21c506166205 (127.12.234.1:32963): tablet 28d3e7ade5064d449360b8ae7aac50c6 (table test-table [id=6db0d36c3e3d4d728452ac762fd25038]) successfully deleted
I20260809 04:44:04.861323 18108 ts_tablet_manager.cc:1918] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:04.861563 18108 log.cc:1196] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-1/wal/wals/28d3e7ade5064d449360b8ae7aac50c6
I20260809 04:44:04.862309 18108 ts_tablet_manager.cc:1939] T 28d3e7ade5064d449360b8ae7aac50c6 P 56edc2c22f004f4fb9f5310786fc1a00: Deleting consensus metadata
I20260809 04:44:04.864676 18109 tablet_replica.cc:330] T 28d3e7ade5064d449360b8ae7aac50c6 P b4d2215b9e674777ad1f904eda053586: stopping tablet replica
I20260809 04:44:04.865142 18109 raft_consensus.cc:2229] T 28d3e7ade5064d449360b8ae7aac50c6 P b4d2215b9e674777ad1f904eda053586 [term 1 FOLLOWER]: Raft consensus shutting down.
Aug 09 04:44:04 dist-test-slave-1jjb krb5kdc[17539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.1@KRBTEST.COM
I20260809 04:44:04.865694 18109 raft_consensus.cc:2258] T 28d3e7ade5064d449360b8ae7aac50c6 P b4d2215b9e674777ad1f904eda053586 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:04.866566 17571 catalog_manager.cc:4841] TS 56edc2c22f004f4fb9f5310786fc1a00 (127.12.234.2:38673): tablet 28d3e7ade5064d449360b8ae7aac50c6 (table test-table [id=6db0d36c3e3d4d728452ac762fd25038]) successfully deleted
I20260809 04:44:04.868186 18109 ts_tablet_manager.cc:1905] T 28d3e7ade5064d449360b8ae7aac50c6 P b4d2215b9e674777ad1f904eda053586: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:04.873591 18109 ts_tablet_manager.cc:1918] T 28d3e7ade5064d449360b8ae7aac50c6 P b4d2215b9e674777ad1f904eda053586: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:04.873790 18109 log.cc:1196] T 28d3e7ade5064d449360b8ae7aac50c6 P b4d2215b9e674777ad1f904eda053586: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786250603326224-13224-0/minicluster-data/ts-2/wal/wals/28d3e7ade5064d449360b8ae7aac50c6
I20260809 04:44:04.874328 18109 ts_tablet_manager.cc:1939] T 28d3e7ade5064d449360b8ae7aac50c6 P b4d2215b9e674777ad1f904eda053586: Deleting consensus metadata
I20260809 04:44:04.874854 18077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:04.840171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52958 (local address 127.12.234.1:32963)
0809 04:44:04.840352 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:04.840359 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:04.850488 (+ 10129us) server_negotiation.cc:407] Connection header received
0809 04:44:04.850771 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:04.850778 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:04.850877 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:04.850991 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:04.855973 (+ 4982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.857387 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.859397 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.860140 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.860246 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:04.866675 (+ 6429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:04.866697 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:04.866717 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:04.866768 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:04.870958 (+ 4190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:04.872031 (+ 1073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:04.872041 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:04.872044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:04.872260 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:04.872746 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:04.872752 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:04.872756 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:04.873236 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:04.873557 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:04.874181 (+ 624us) server_negotiation.cc:299] Negotiation successful
0809 04:44:04.874452 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:44:04.876202 17568 catalog_manager.cc:4841] TS b4d2215b9e674777ad1f904eda053586 (127.12.234.3:44199): tablet 28d3e7ade5064d449360b8ae7aac50c6 (table test-table [id=6db0d36c3e3d4d728452ac762fd25038]) successfully deleted
I20260809 04:44:04.876456 17779 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:04.966434 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 17650
I20260809 04:44:04.988456 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 17790
Aug 09 04:44:04 dist-test-slave-1jjb krb5kdc[17539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.2@KRBTEST.COM
I20260809 04:44:05.005375 18078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:04.989211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33382 (local address 127.12.234.2:38673)
0809 04:44:04.989393 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:04.989400 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:04.989531 (+ 131us) server_negotiation.cc:407] Connection header received
0809 04:44:04.989731 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:04.989736 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:04.989838 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:04.989957 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:04.991141 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.991984 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.993230 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:04.993693 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:04.993856 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:04.998443 (+ 4587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:04.998454 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:04.998470 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:04.998504 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:05.002555 (+ 4051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:05.003195 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:05.003205 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:05.003209 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:05.003391 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:05.003741 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:05.003747 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:05.003750 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:05.004237 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:05.004582 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:05.004740 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 04:44:05.004974 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:44:05.006188 17918 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:05.122097 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 17790
I20260809 04:44:05.150332 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 17928
Aug 09 04:44:05 dist-test-slave-1jjb krb5kdc[17539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.3@KRBTEST.COM
I20260809 04:44:05.169420 18076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:05.151151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38400 (local address 127.12.234.3:44199)
0809 04:44:05.151338 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:05.151343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:05.151362 (+ 19us) server_negotiation.cc:407] Connection header received
0809 04:44:05.151517 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:05.151521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:05.151597 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:05.151688 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:05.153626 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:05.154406 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:05.155980 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:05.156605 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:05.156716 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:05.162121 (+ 5405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:05.162139 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:05.162159 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:05.162202 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:05.165246 (+ 3044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:05.166184 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:05.166212 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:05.166214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:05.166368 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:05.167091 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:05.167127 (+ 36us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:05.167130 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:05.167712 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:05.168027 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:05.168701 (+ 674us) server_negotiation.cc:299] Negotiation successful
0809 04:44:05.169014 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 04:44:05.170202 18057 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:05.267655 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 17928
I20260809 04:44:05.288970 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 17553
I20260809 04:44:05.290100 17617 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:05.399314 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 17553
2026-08-09T04:44:05Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4467 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T04:44:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:05Z Disabled control of system clock
I20260809 04:44:05.481393 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:46841
--webserver_interface=127.12.234.62
--webserver_port=0
--builtin_ntp_servers=127.12.234.20:45003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:46841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 04:44:05.841248 18119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:05.841593 18119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:05.841749 18119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:05.849699 18119 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:05.849840 18119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:05.849913 18119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:05.850006 18119 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:05.850087 18119 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:05.860589 18119 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:45003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:46841
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:46841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:05.862106 18119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:05.864192 18119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:05.877463 18124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:05.877596 18125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:05.877646 18128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:05.879280 18119 server_base.cc:1034] running on GCE node
I20260809 04:44:05.880190 18119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:05.881713 18119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:05.882936 18119 hybrid_clock.cc:648] HybridClock initialized: now 1786250645882871 us; error 70 us; skew 500 ppm
I20260809 04:44:05.883342 18119 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:44:05.886040 18119 webserver.cc:460] Webserver started at http://127.12.234.62:44065/ using document root <none> and password file <none>
I20260809 04:44:05.886816 18119 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:05.886907 18119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:05.887300 18119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:05.890374 18119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "8fed6469b3484b80b89d13b0598c88bb"
format_stamp: "Formatted at 2026-08-09 04:44:05 on dist-test-slave-1jjb"
server_key: "70fb2d8646e9646e3d69e815a94cb954"
server_key_iv: "84d887993597f9c60de5ccbe5c335fac"
server_key_version: "encryptionkey@0"
I20260809 04:44:05.891207 18119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "8fed6469b3484b80b89d13b0598c88bb"
format_stamp: "Formatted at 2026-08-09 04:44:05 on dist-test-slave-1jjb"
server_key: "70fb2d8646e9646e3d69e815a94cb954"
server_key_iv: "84d887993597f9c60de5ccbe5c335fac"
server_key_version: "encryptionkey@0"
I20260809 04:44:05.897058 18119 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 04:44:05.901957 18134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:05.903757 18119 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:44:05.903971 18119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "8fed6469b3484b80b89d13b0598c88bb"
format_stamp: "Formatted at 2026-08-09 04:44:05 on dist-test-slave-1jjb"
server_key: "70fb2d8646e9646e3d69e815a94cb954"
server_key_iv: "84d887993597f9c60de5ccbe5c335fac"
server_key_version: "encryptionkey@0"
I20260809 04:44:05.904245 18119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:05.910600 18119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:05.911484 18119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:05.911830 18119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:05.933706 18119 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:46841
I20260809 04:44:05.933955 18186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:46841 every 8 connection(s)
I20260809 04:44:05.935329 18119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:05.937695 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 18119
I20260809 04:44:05.937950 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:44:05.938941 13224 external_mini_cluster.cc:1428] Setting key 5ad107ac6cc34e441743c23f8366937e
I20260809 04:44:05.941963 18187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:05.949479 18190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:05.941869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33976 (local address 127.12.234.62:46841)
0809 04:44:05.943225 (+ 1356us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:05.943240 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:05.943293 (+ 53us) server_negotiation.cc:407] Connection header received
0809 04:44:05.945251 (+ 1958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:05.945280 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:05.945900 (+ 620us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:05.946357 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:05.947578 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:05.947585 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:05.947598 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:05.947619 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:05.947701 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:05.947717 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:05.948019 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:05.948194 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 04:44:05.948469 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":749,"thread_start_us":447,"threads_started":1}
I20260809 04:44:05.955156 18187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fed6469b3484b80b89d13b0598c88bb: Bootstrap starting.
I20260809 04:44:05.958992 18187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fed6469b3484b80b89d13b0598c88bb: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:05.960214 18187 log.cc:824] T 00000000000000000000000000000000 P 8fed6469b3484b80b89d13b0598c88bb: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:05.963335 18187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fed6469b3484b80b89d13b0598c88bb: No bootstrap required, opened a new log
I20260809 04:44:05.971576 18187 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fed6469b3484b80b89d13b0598c88bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fed6469b3484b80b89d13b0598c88bb" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46841 } }
I20260809 04:44:05.971990 18187 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fed6469b3484b80b89d13b0598c88bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:05.972129 18187 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fed6469b3484b80b89d13b0598c88bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fed6469b3484b80b89d13b0598c88bb, State: Initialized, Role: FOLLOWER
I20260809 04:44:05.972822 18187 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fed6469b3484b80b89d13b0598c88bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fed6469b3484b80b89d13b0598c88bb" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46841 } }
I20260809 04:44:05.973121 18187 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fed6469b3484b80b89d13b0598c88bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:05.973307 18187 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fed6469b3484b80b89d13b0598c88bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:05.973500 18187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fed6469b3484b80b89d13b0598c88bb [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:05.975812 18187 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fed6469b3484b80b89d13b0598c88bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fed6469b3484b80b89d13b0598c88bb" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46841 } }
I20260809 04:44:05.976444 18187 leader_election.cc:302] T 00000000000000000000000000000000 P 8fed6469b3484b80b89d13b0598c88bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fed6469b3484b80b89d13b0598c88bb; no voters:
I20260809 04:44:05.977097 18187 leader_election.cc:288] T 00000000000000000000000000000000 P 8fed6469b3484b80b89d13b0598c88bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:05.977259 18192 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fed6469b3484b80b89d13b0598c88bb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:05.978022 18192 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fed6469b3484b80b89d13b0598c88bb [term 1 LEADER]: Becoming Leader. State: Replica: 8fed6469b3484b80b89d13b0598c88bb, State: Running, Role: LEADER
I20260809 04:44:05.978609 18192 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fed6469b3484b80b89d13b0598c88bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fed6469b3484b80b89d13b0598c88bb" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46841 } }
I20260809 04:44:05.979388 18187 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fed6469b3484b80b89d13b0598c88bb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:05.981946 18193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fed6469b3484b80b89d13b0598c88bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8fed6469b3484b80b89d13b0598c88bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fed6469b3484b80b89d13b0598c88bb" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46841 } } }
I20260809 04:44:05.981937 18194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fed6469b3484b80b89d13b0598c88bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fed6469b3484b80b89d13b0598c88bb. Latest consensus state: current_term: 1 leader_uuid: "8fed6469b3484b80b89d13b0598c88bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fed6469b3484b80b89d13b0598c88bb" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46841 } } }
I20260809 04:44:05.982463 18193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fed6469b3484b80b89d13b0598c88bb [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:05.982465 18194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fed6469b3484b80b89d13b0598c88bb [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:05.983806 18199 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:05.989732 18199 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:05.999066 18199 catalog_manager.cc:1347] Generated new cluster ID: 62a3c7b9abdb4e28b11719be0f0adcfb
I20260809 04:44:05.999207 18199 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:06.016378 18199 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:06.017076 18199 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:06.025768 18199 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fed6469b3484b80b89d13b0598c88bb: Generated new TSK 0
I20260809 04:44:06.026968 18199 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:44:06.039943 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:0
--local_ip_for_outbound_sockets=127.12.234.1
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:46841
--builtin_ntp_servers=127.12.234.20:45003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 04:44:06.385171 18212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:06.385511 18212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:06.385685 18212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:06.393743 18212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:06.393831 18212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:06.393965 18212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:44:06.403187 18212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:45003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:46841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:06.404551 18212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:06.406315 18212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:06.419191 18218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:06.419346 18219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:06.419366 18221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:06.419991 18212 server_base.cc:1034] running on GCE node
I20260809 04:44:06.420559 18212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:06.421538 18212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:06.422725 18212 hybrid_clock.cc:648] HybridClock initialized: now 1786250646422699 us; error 28 us; skew 500 ppm
I20260809 04:44:06.423009 18212 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:44:06.425175 18212 webserver.cc:460] Webserver started at http://127.12.234.1:34981/ using document root <none> and password file <none>
I20260809 04:44:06.425863 18212 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:06.425982 18212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:06.426324 18212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:06.429024 18212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-0/data/instance:
uuid: "6e0e6820452e421b9294f8a0de94b2b6"
format_stamp: "Formatted at 2026-08-09 04:44:06 on dist-test-slave-1jjb"
server_key: "6d5d36777dabe73364b1dd7c7965302c"
server_key_iv: "9cb6719aecc2b36f37b5b949a61cb262"
server_key_version: "encryptionkey@0"
I20260809 04:44:06.429824 18212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance:
uuid: "6e0e6820452e421b9294f8a0de94b2b6"
format_stamp: "Formatted at 2026-08-09 04:44:06 on dist-test-slave-1jjb"
server_key: "6d5d36777dabe73364b1dd7c7965302c"
server_key_iv: "9cb6719aecc2b36f37b5b949a61cb262"
server_key_version: "encryptionkey@0"
I20260809 04:44:06.434526 18212 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:44:06.437803 18227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:06.439136 18212 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 04:44:06.439284 18212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "6e0e6820452e421b9294f8a0de94b2b6"
format_stamp: "Formatted at 2026-08-09 04:44:06 on dist-test-slave-1jjb"
server_key: "6d5d36777dabe73364b1dd7c7965302c"
server_key_iv: "9cb6719aecc2b36f37b5b949a61cb262"
server_key_version: "encryptionkey@0"
I20260809 04:44:06.439453 18212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:06.444669 18212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:06.445358 18212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:06.445585 18212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:06.446393 18212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:06.447621 18212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:06.447700 18212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:06.447774 18212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:06.447824 18212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:06.480058 18212 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:33017
I20260809 04:44:06.480145 18344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:33017 every 8 connection(s)
I20260809 04:44:06.481372 18212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:44:06.483548 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 18212
I20260809 04:44:06.483777 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance
I20260809 04:44:06.484566 13224 external_mini_cluster.cc:1428] Setting key 47771c5d5781cd194e9bf756534f1a06
I20260809 04:44:06.487982 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:0
--local_ip_for_outbound_sockets=127.12.234.2
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:46841
--builtin_ntp_servers=127.12.234.20:45003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 04:44:06.490762 18347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:06.483956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:37547 (local address 127.12.234.62:46841)
0809 04:44:06.488392 (+ 4436us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:06.488396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:06.488410 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:44:06.488449 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:06.488452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:06.488491 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:06.488623 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:06.489955 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:06.489961 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:06.489965 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:06.489984 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:06.490041 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:06.490053 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:06.490176 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:06.490417 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 04:44:06.490478 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4339,"thread_start_us":4261,"threads_started":1}
I20260809 04:44:06.491526 18348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:06.484731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46841 (local address 127.12.234.1:37547)
0809 04:44:06.485958 (+ 1227us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:06.486016 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:06.487615 (+ 1599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:06.488864 (+ 1249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:06.488872 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:06.488955 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:06.489672 (+ 717us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:44:06.489675 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:06.489710 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:44:06.489790 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:06.490152 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:06.490163 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:06.490171 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 04:44:06.490459 (+ 288us) client_negotiation.cc:240] Negotiation successful
0809 04:44:06.490704 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":374,"threads_started":1}
I20260809 04:44:06.492941 18345 heartbeater.cc:347] Connected to a master server at 127.12.234.62:46841
I20260809 04:44:06.493232 18345 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:06.493749 18345 heartbeater.cc:511] Master 127.12.234.62:46841 requested a full tablet report, sending...
I20260809 04:44:06.495693 18151 ts_manager.cc:194] Registered new tserver with Master: 6e0e6820452e421b9294f8a0de94b2b6 (127.12.234.1:33017)
W20260809 04:44:06.838593 18349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:06.838882 18349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:06.839035 18349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:06.846796 18349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:06.846903 18349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:06.847108 18349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:44:06.856511 18349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:45003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:46841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:06.857815 18349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:06.859549 18349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:06.872913 18357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:06.872884 18355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:06.872959 18354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:06.873757 18349 server_base.cc:1034] running on GCE node
I20260809 04:44:06.874392 18349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:06.875285 18349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:06.876531 18349 hybrid_clock.cc:648] HybridClock initialized: now 1786250646876484 us; error 59 us; skew 500 ppm
I20260809 04:44:06.876852 18349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:44:06.878974 18349 webserver.cc:460] Webserver started at http://127.12.234.2:41851/ using document root <none> and password file <none>
I20260809 04:44:06.879689 18349 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:06.879832 18349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:06.880204 18349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:06.882982 18349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-1/data/instance:
uuid: "67117d7b7c1d41b89eeed7d826143c1f"
format_stamp: "Formatted at 2026-08-09 04:44:06 on dist-test-slave-1jjb"
server_key: "24e22dcd84221ba00c2fdbefd8b8f8a0"
server_key_iv: "ef6aba6905857eae2862fe936cba9448"
server_key_version: "encryptionkey@0"
I20260809 04:44:06.883710 18349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance:
uuid: "67117d7b7c1d41b89eeed7d826143c1f"
format_stamp: "Formatted at 2026-08-09 04:44:06 on dist-test-slave-1jjb"
server_key: "24e22dcd84221ba00c2fdbefd8b8f8a0"
server_key_iv: "ef6aba6905857eae2862fe936cba9448"
server_key_version: "encryptionkey@0"
I20260809 04:44:06.888779 18349 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260809 04:44:06.892063 18364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:06.893366 18349 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:44:06.893489 18349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "67117d7b7c1d41b89eeed7d826143c1f"
format_stamp: "Formatted at 2026-08-09 04:44:06 on dist-test-slave-1jjb"
server_key: "24e22dcd84221ba00c2fdbefd8b8f8a0"
server_key_iv: "ef6aba6905857eae2862fe936cba9448"
server_key_version: "encryptionkey@0"
I20260809 04:44:06.893661 18349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:06.899259 18349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:06.899977 18349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:06.900278 18349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:06.901106 18349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:06.902594 18349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:06.902681 18349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:06.902761 18349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:06.902817 18349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:06.936007 18349 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:42333
I20260809 04:44:06.936040 18482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:42333 every 8 connection(s)
I20260809 04:44:06.937405 18349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:44:06.943010 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 18349
I20260809 04:44:06.943181 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance
I20260809 04:44:06.943742 13224 external_mini_cluster.cc:1428] Setting key 0ec807e7ae08318a2605f1c5f292d28a
I20260809 04:44:06.946194 18347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:06.940002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:54241 (local address 127.12.234.62:46841)
0809 04:44:06.940200 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:06.940204 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:06.941687 (+ 1483us) server_negotiation.cc:407] Connection header received
0809 04:44:06.943642 (+ 1955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:06.943647 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:06.943714 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:06.943821 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:06.945354 (+ 1533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:06.945360 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:06.945381 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:06.945401 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:06.945463 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:06.945483 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:06.945612 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:06.945852 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 04:44:06.945914 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:44:06.946731 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:0
--local_ip_for_outbound_sockets=127.12.234.3
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:46841
--builtin_ntp_servers=127.12.234.20:45003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 04:44:06.947207 18485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:06.940638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46841 (local address 127.12.234.2:54241)
0809 04:44:06.941534 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:06.941574 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:06.943311 (+ 1737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:06.944147 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:06.944154 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:06.944235 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:06.944992 (+ 757us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:44:06.944996 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:06.945050 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:44:06.945159 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:06.945601 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:06.945614 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:06.945625 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 04:44:06.945908 (+ 283us) client_negotiation.cc:240] Negotiation successful
0809 04:44:06.946144 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":286,"threads_started":1}
I20260809 04:44:06.948523 18483 heartbeater.cc:347] Connected to a master server at 127.12.234.62:46841
I20260809 04:44:06.948832 18483 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:06.949534 18483 heartbeater.cc:511] Master 127.12.234.62:46841 requested a full tablet report, sending...
I20260809 04:44:06.951045 18151 ts_manager.cc:194] Registered new tserver with Master: 67117d7b7c1d41b89eeed7d826143c1f (127.12.234.2:42333)
W20260809 04:44:07.283103 18487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:07.283396 18487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:07.283573 18487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:07.290990 18487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:07.291112 18487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:07.291301 18487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:44:07.300916 18487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:45003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:46841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:07.302269 18487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:07.304032 18487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:07.317230 18493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:07.317320 18492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:07.317320 18495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:07.318696 18487 server_base.cc:1034] running on GCE node
I20260809 04:44:07.319353 18487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:07.320381 18487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:07.321631 18487 hybrid_clock.cc:648] HybridClock initialized: now 1786250647321581 us; error 52 us; skew 500 ppm
I20260809 04:44:07.322042 18487 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:44:07.324433 18487 webserver.cc:460] Webserver started at http://127.12.234.3:37497/ using document root <none> and password file <none>
I20260809 04:44:07.325130 18487 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:07.325261 18487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:07.325616 18487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:07.328554 18487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-2/data/instance:
uuid: "5a90fa2f65624f8382dceb89d56cae23"
format_stamp: "Formatted at 2026-08-09 04:44:07 on dist-test-slave-1jjb"
server_key: "a370502c6e406e148fccf4de5f032c09"
server_key_iv: "d2c405c9411aa126d40743bd90994330"
server_key_version: "encryptionkey@0"
I20260809 04:44:07.329478 18487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance:
uuid: "5a90fa2f65624f8382dceb89d56cae23"
format_stamp: "Formatted at 2026-08-09 04:44:07 on dist-test-slave-1jjb"
server_key: "a370502c6e406e148fccf4de5f032c09"
server_key_iv: "d2c405c9411aa126d40743bd90994330"
server_key_version: "encryptionkey@0"
I20260809 04:44:07.335438 18487 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260809 04:44:07.339128 18501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:07.340570 18487 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:44:07.340749 18487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "5a90fa2f65624f8382dceb89d56cae23"
format_stamp: "Formatted at 2026-08-09 04:44:07 on dist-test-slave-1jjb"
server_key: "a370502c6e406e148fccf4de5f032c09"
server_key_iv: "d2c405c9411aa126d40743bd90994330"
server_key_version: "encryptionkey@0"
I20260809 04:44:07.340974 18487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:07.347222 18487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:07.347954 18487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:07.348227 18487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:07.349184 18487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:07.351097 18487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:07.351222 18487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:07.351372 18487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:07.351475 18487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:07.385958 18487 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:42559
I20260809 04:44:07.386008 18619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:42559 every 8 connection(s)
I20260809 04:44:07.387249 18487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:44:07.390568 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 18487
I20260809 04:44:07.390754 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance
I20260809 04:44:07.391185 13224 external_mini_cluster.cc:1428] Setting key 895a7a06446a443ea5e6def475290623
I20260809 04:44:07.395718 18347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:07.389799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:45037 (local address 127.12.234.62:46841)
0809 04:44:07.389971 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:07.389975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:07.391498 (+ 1523us) server_negotiation.cc:407] Connection header received
0809 04:44:07.393568 (+ 2070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:07.393573 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:07.393644 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:07.393747 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:07.395121 (+ 1374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:07.395125 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:07.395129 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:07.395144 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:07.395173 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:07.395184 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:07.395293 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:07.395467 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 04:44:07.395515 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 04:44:07.396657 18622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:07.390330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46841 (local address 127.12.234.3:45037)
0809 04:44:07.391335 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:07.391375 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:07.393324 (+ 1949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:07.394031 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:07.394046 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:07.394180 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:07.394842 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:44:07.394845 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:07.394881 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:44:07.394961 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:07.395284 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:07.395291 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:07.395297 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 04:44:07.395508 (+ 211us) client_negotiation.cc:240] Negotiation successful
0809 04:44:07.395691 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":341,"threads_started":1}
I20260809 04:44:07.398119 18620 heartbeater.cc:347] Connected to a master server at 127.12.234.62:46841
I20260809 04:44:07.398416 18620 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:07.398953 18620 heartbeater.cc:511] Master 127.12.234.62:46841 requested a full tablet report, sending...
I20260809 04:44:07.400254 18151 ts_manager.cc:194] Registered new tserver with Master: 5a90fa2f65624f8382dceb89d56cae23 (127.12.234.3:42559)
I20260809 04:44:07.406104 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:44:07.413105 18347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:07.410773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33980 (local address 127.12.234.62:46841)
0809 04:44:07.410918 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:07.410922 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:07.411466 (+ 544us) server_negotiation.cc:407] Connection header received
0809 04:44:07.411594 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:07.411598 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:07.411644 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:07.411720 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:07.412618 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:07.412622 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:07.412625 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:07.412637 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:07.412663 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:07.412673 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:07.412763 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:07.412853 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 04:44:07.412897 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 04:44:07.416903 18151 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33980:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:44:07.423496 18151 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:44:07.441468 18634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:07.434728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:33017 (local address 127.0.0.1:36006)
0809 04:44:07.436683 (+ 1955us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:07.436739 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:07.436900 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:07.438763 (+ 1863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:07.438774 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:07.438878 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:07.439826 (+ 948us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:44:07.439832 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:07.439867 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:44:07.439960 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:07.440641 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:07.440656 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:07.440668 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 04:44:07.440871 (+ 203us) client_negotiation.cc:240] Negotiation successful
0809 04:44:07.440978 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1643,"threads_started":1}
I20260809 04:44:07.442440 18633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:07.435033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36006 (local address 127.12.234.1:33017)
0809 04:44:07.435788 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:07.435802 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:07.437073 (+ 1271us) server_negotiation.cc:407] Connection header received
0809 04:44:07.437167 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:07.437179 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:07.437452 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:07.437616 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:07.440239 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:07.440252 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:07.440271 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:07.440295 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:07.440373 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:07.440404 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:07.440580 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:07.441840 (+ 1260us) server_negotiation.cc:299] Negotiation successful
0809 04:44:07.441972 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":1268,"threads_started":1}
I20260809 04:44:07.447571 18635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:07.435942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35352 (local address 127.12.234.3:42559)
0809 04:44:07.444824 (+ 8882us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:07.444841 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:07.444876 (+ 35us) server_negotiation.cc:407] Connection header received
0809 04:44:07.444986 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:07.444996 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:07.445291 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:07.445475 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:07.446736 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:07.446745 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:07.446758 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:07.446775 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:07.446825 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:07.446843 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:07.447052 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:07.447122 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 04:44:07.447212 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8699,"thread_start_us":8516,"threads_started":1}
I20260809 04:44:07.447638 18636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:07.435098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:42559 (local address 127.0.0.1:35352)
0809 04:44:07.443698 (+ 8600us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:07.443717 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:07.443819 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:07.445511 (+ 1692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:07.445517 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:07.445541 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:07.446524 (+ 983us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:44:07.446528 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:07.446545 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:44:07.446587 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:07.446967 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:07.446974 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:07.446982 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 04:44:07.447153 (+ 171us) client_negotiation.cc:240] Negotiation successful
0809 04:44:07.447207 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4543,"thread_start_us":874,"threads_started":1}
I20260809 04:44:07.448230 18634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:07.436965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:42333 (local address 127.0.0.1:34144)
0809 04:44:07.443048 (+ 6083us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:07.443071 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:07.443168 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:07.445768 (+ 2600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:07.445774 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:07.445804 (+ 30us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:07.446323 (+ 519us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:44:07.446327 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:07.446348 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:44:07.446403 (+ 55us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:07.447788 (+ 1385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:07.447795 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:07.447802 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 04:44:07.447891 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 04:44:07.447936 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5915,"thread_start_us":2733,"threads_started":1}
I20260809 04:44:07.449375 18637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:07.437750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34144 (local address 127.12.234.2:42333)
0809 04:44:07.445150 (+ 7400us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:07.445205 (+ 55us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:07.445236 (+ 31us) server_negotiation.cc:407] Connection header received
0809 04:44:07.445322 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:07.445327 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:07.445545 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:07.446176 (+ 631us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:07.447521 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:07.447530 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:07.447542 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:07.447567 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:07.447625 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:07.447649 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:07.448831 (+ 1182us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:07.448941 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 04:44:07.449029 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7247,"thread_start_us":7094,"threads_started":1}
I20260809 04:44:07.451143 18552 tablet_service.cc:1467] Processing CreateTablet for tablet b8e4c1bf9e7347cab50aad4c7d31182f (DEFAULT_TABLE table=test-table [id=f4963378ff98492882326af9a7ef4ca3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:07.449187 18276 tablet_service.cc:1467] Processing CreateTablet for tablet b8e4c1bf9e7347cab50aad4c7d31182f (DEFAULT_TABLE table=test-table [id=f4963378ff98492882326af9a7ef4ca3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:07.452111 18416 tablet_service.cc:1467] Processing CreateTablet for tablet b8e4c1bf9e7347cab50aad4c7d31182f (DEFAULT_TABLE table=test-table [id=f4963378ff98492882326af9a7ef4ca3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:07.452636 18552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8e4c1bf9e7347cab50aad4c7d31182f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:07.453169 18276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8e4c1bf9e7347cab50aad4c7d31182f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:07.453480 18416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8e4c1bf9e7347cab50aad4c7d31182f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:07.465341 18638 tablet_bootstrap.cc:492] T b8e4c1bf9e7347cab50aad4c7d31182f P 67117d7b7c1d41b89eeed7d826143c1f: Bootstrap starting.
I20260809 04:44:07.467384 18639 tablet_bootstrap.cc:492] T b8e4c1bf9e7347cab50aad4c7d31182f P 6e0e6820452e421b9294f8a0de94b2b6: Bootstrap starting.
I20260809 04:44:07.468287 18640 tablet_bootstrap.cc:492] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23: Bootstrap starting.
I20260809 04:44:07.470762 18638 tablet_bootstrap.cc:654] T b8e4c1bf9e7347cab50aad4c7d31182f P 67117d7b7c1d41b89eeed7d826143c1f: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:07.471138 18639 tablet_bootstrap.cc:654] T b8e4c1bf9e7347cab50aad4c7d31182f P 6e0e6820452e421b9294f8a0de94b2b6: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:07.471493 18640 tablet_bootstrap.cc:654] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:07.472146 18638 log.cc:824] T b8e4c1bf9e7347cab50aad4c7d31182f P 67117d7b7c1d41b89eeed7d826143c1f: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:07.472507 18639 log.cc:824] T b8e4c1bf9e7347cab50aad4c7d31182f P 6e0e6820452e421b9294f8a0de94b2b6: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:07.472827 18640 log.cc:824] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:07.481120 18640 tablet_bootstrap.cc:492] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23: No bootstrap required, opened a new log
I20260809 04:44:07.481449 18640 ts_tablet_manager.cc:1397] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.007s
I20260809 04:44:07.481879 18638 tablet_bootstrap.cc:492] T b8e4c1bf9e7347cab50aad4c7d31182f P 67117d7b7c1d41b89eeed7d826143c1f: No bootstrap required, opened a new log
I20260809 04:44:07.482225 18638 ts_tablet_manager.cc:1397] T b8e4c1bf9e7347cab50aad4c7d31182f P 67117d7b7c1d41b89eeed7d826143c1f: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.004s
I20260809 04:44:07.485136 18639 tablet_bootstrap.cc:492] T b8e4c1bf9e7347cab50aad4c7d31182f P 6e0e6820452e421b9294f8a0de94b2b6: No bootstrap required, opened a new log
I20260809 04:44:07.485535 18639 ts_tablet_manager.cc:1397] T b8e4c1bf9e7347cab50aad4c7d31182f P 6e0e6820452e421b9294f8a0de94b2b6: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.004s
I20260809 04:44:07.492652 18638 raft_consensus.cc:348] T b8e4c1bf9e7347cab50aad4c7d31182f P 67117d7b7c1d41b89eeed7d826143c1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0e6820452e421b9294f8a0de94b2b6" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 33017 } } peers { permanent_uuid: "5a90fa2f65624f8382dceb89d56cae23" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42559 } } peers { permanent_uuid: "67117d7b7c1d41b89eeed7d826143c1f" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42333 } }
I20260809 04:44:07.492652 18640 raft_consensus.cc:348] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0e6820452e421b9294f8a0de94b2b6" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 33017 } } peers { permanent_uuid: "5a90fa2f65624f8382dceb89d56cae23" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42559 } } peers { permanent_uuid: "67117d7b7c1d41b89eeed7d826143c1f" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42333 } }
I20260809 04:44:07.493086 18640 raft_consensus.cc:374] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:07.493132 18638 raft_consensus.cc:374] T b8e4c1bf9e7347cab50aad4c7d31182f P 67117d7b7c1d41b89eeed7d826143c1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:07.493245 18640 raft_consensus.cc:729] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a90fa2f65624f8382dceb89d56cae23, State: Initialized, Role: FOLLOWER
I20260809 04:44:07.493232 18638 raft_consensus.cc:729] T b8e4c1bf9e7347cab50aad4c7d31182f P 67117d7b7c1d41b89eeed7d826143c1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67117d7b7c1d41b89eeed7d826143c1f, State: Initialized, Role: FOLLOWER
I20260809 04:44:07.493875 18640 consensus_queue.cc:260] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0e6820452e421b9294f8a0de94b2b6" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 33017 } } peers { permanent_uuid: "5a90fa2f65624f8382dceb89d56cae23" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42559 } } peers { permanent_uuid: "67117d7b7c1d41b89eeed7d826143c1f" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42333 } }
I20260809 04:44:07.493875 18638 consensus_queue.cc:260] T b8e4c1bf9e7347cab50aad4c7d31182f P 67117d7b7c1d41b89eeed7d826143c1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0e6820452e421b9294f8a0de94b2b6" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 33017 } } peers { permanent_uuid: "5a90fa2f65624f8382dceb89d56cae23" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42559 } } peers { permanent_uuid: "67117d7b7c1d41b89eeed7d826143c1f" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42333 } }
I20260809 04:44:07.495040 18483 heartbeater.cc:503] Master 127.12.234.62:46841 was elected leader, sending a full tablet report...
I20260809 04:44:07.495932 18638 ts_tablet_manager.cc:1428] T b8e4c1bf9e7347cab50aad4c7d31182f P 67117d7b7c1d41b89eeed7d826143c1f: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260809 04:44:07.496963 18620 heartbeater.cc:503] Master 127.12.234.62:46841 was elected leader, sending a full tablet report...
I20260809 04:44:07.497465 18640 ts_tablet_manager.cc:1428] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23: Time spent starting tablet: real 0.016s user 0.012s sys 0.002s
I20260809 04:44:07.498420 18345 heartbeater.cc:503] Master 127.12.234.62:46841 was elected leader, sending a full tablet report...
I20260809 04:44:07.498464 18639 raft_consensus.cc:348] T b8e4c1bf9e7347cab50aad4c7d31182f P 6e0e6820452e421b9294f8a0de94b2b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0e6820452e421b9294f8a0de94b2b6" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 33017 } } peers { permanent_uuid: "5a90fa2f65624f8382dceb89d56cae23" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42559 } } peers { permanent_uuid: "67117d7b7c1d41b89eeed7d826143c1f" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42333 } }
I20260809 04:44:07.498847 18639 raft_consensus.cc:374] T b8e4c1bf9e7347cab50aad4c7d31182f P 6e0e6820452e421b9294f8a0de94b2b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:07.498968 18639 raft_consensus.cc:729] T b8e4c1bf9e7347cab50aad4c7d31182f P 6e0e6820452e421b9294f8a0de94b2b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e0e6820452e421b9294f8a0de94b2b6, State: Initialized, Role: FOLLOWER
I20260809 04:44:07.499526 18639 consensus_queue.cc:260] T b8e4c1bf9e7347cab50aad4c7d31182f P 6e0e6820452e421b9294f8a0de94b2b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0e6820452e421b9294f8a0de94b2b6" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 33017 } } peers { permanent_uuid: "5a90fa2f65624f8382dceb89d56cae23" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42559 } } peers { permanent_uuid: "67117d7b7c1d41b89eeed7d826143c1f" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42333 } }
I20260809 04:44:07.501950 18639 ts_tablet_manager.cc:1428] T b8e4c1bf9e7347cab50aad4c7d31182f P 6e0e6820452e421b9294f8a0de94b2b6: Time spent starting tablet: real 0.016s user 0.011s sys 0.002s
I20260809 04:44:07.582481 18645 raft_consensus.cc:482] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:44:07.582763 18645 raft_consensus.cc:504] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0e6820452e421b9294f8a0de94b2b6" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 33017 } } peers { permanent_uuid: "5a90fa2f65624f8382dceb89d56cae23" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42559 } } peers { permanent_uuid: "67117d7b7c1d41b89eeed7d826143c1f" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42333 } }
I20260809 04:44:07.584606 18645 leader_election.cc:288] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e0e6820452e421b9294f8a0de94b2b6 (127.12.234.1:33017), 67117d7b7c1d41b89eeed7d826143c1f (127.12.234.2:42333)
I20260809 04:44:07.587347 18637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:07.585317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:49205 (local address 127.12.234.2:42333)
0809 04:44:07.585503 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:07.585507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:07.585522 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:44:07.585572 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:07.585576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:07.585619 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:07.585734 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:07.586723 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:07.586728 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:07.586731 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:07.586745 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:07.586774 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:07.586785 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:07.586922 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:07.587044 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 04:44:07.587100 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 04:44:07.587338 18622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:07.584909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:42333 (local address 127.12.234.3:49205)
0809 04:44:07.585107 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:07.585132 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:07.585242 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:07.585776 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:07.585782 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:07.585806 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:07.586509 (+ 703us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:44:07.586514 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:07.586530 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:44:07.586573 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:07.586899 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:07.586903 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:07.586909 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 04:44:07.587033 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 04:44:07.587087 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 04:44:07.587786 18633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:07.585014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:44439 (local address 127.12.234.1:33017)
0809 04:44:07.585199 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:07.585207 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:07.585945 (+ 738us) server_negotiation.cc:407] Connection header received
0809 04:44:07.586062 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:07.586071 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:07.586132 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:07.586205 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:07.587166 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:07.587172 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:07.587176 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:07.587200 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:07.587227 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:07.587237 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:07.587355 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:07.587501 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 04:44:07.587562 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 04:44:07.588428 18647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:07.584999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:33017 (local address 127.12.234.3:44439)
0809 04:44:07.585835 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:07.585851 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:07.585945 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:07.586271 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:07.586279 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:07.586306 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:07.586968 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:44:07.586970 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:07.586983 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:44:07.587017 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:07.587336 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:07.587341 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:07.587345 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 04:44:07.588001 (+ 656us) client_negotiation.cc:240] Negotiation successful
0809 04:44:07.588053 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"thread_start_us":506,"threads_started":1}
I20260809 04:44:07.588385 18436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e4c1bf9e7347cab50aad4c7d31182f" candidate_uuid: "5a90fa2f65624f8382dceb89d56cae23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67117d7b7c1d41b89eeed7d826143c1f" is_pre_election: true
I20260809 04:44:07.588896 18436 raft_consensus.cc:2454] T b8e4c1bf9e7347cab50aad4c7d31182f P 67117d7b7c1d41b89eeed7d826143c1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a90fa2f65624f8382dceb89d56cae23 in term 0.
I20260809 04:44:07.589668 18504 leader_election.cc:302] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a90fa2f65624f8382dceb89d56cae23, 67117d7b7c1d41b89eeed7d826143c1f; no voters:
I20260809 04:44:07.589534 18298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e4c1bf9e7347cab50aad4c7d31182f" candidate_uuid: "5a90fa2f65624f8382dceb89d56cae23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0e6820452e421b9294f8a0de94b2b6" is_pre_election: true
I20260809 04:44:07.590046 18645 raft_consensus.cc:2781] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:44:07.590193 18645 raft_consensus.cc:482] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:44:07.590104 18298 raft_consensus.cc:2454] T b8e4c1bf9e7347cab50aad4c7d31182f P 6e0e6820452e421b9294f8a0de94b2b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a90fa2f65624f8382dceb89d56cae23 in term 0.
I20260809 04:44:07.590325 18645 raft_consensus.cc:3037] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:07.592724 18645 raft_consensus.cc:504] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0e6820452e421b9294f8a0de94b2b6" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 33017 } } peers { permanent_uuid: "5a90fa2f65624f8382dceb89d56cae23" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42559 } } peers { permanent_uuid: "67117d7b7c1d41b89eeed7d826143c1f" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42333 } }
I20260809 04:44:07.593288 18645 leader_election.cc:288] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23 [CANDIDATE]: Term 1 election: Requested vote from peers 6e0e6820452e421b9294f8a0de94b2b6 (127.12.234.1:33017), 67117d7b7c1d41b89eeed7d826143c1f (127.12.234.2:42333)
I20260809 04:44:07.593703 18298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e4c1bf9e7347cab50aad4c7d31182f" candidate_uuid: "5a90fa2f65624f8382dceb89d56cae23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e0e6820452e421b9294f8a0de94b2b6"
I20260809 04:44:07.593717 18436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8e4c1bf9e7347cab50aad4c7d31182f" candidate_uuid: "5a90fa2f65624f8382dceb89d56cae23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67117d7b7c1d41b89eeed7d826143c1f"
I20260809 04:44:07.593895 18298 raft_consensus.cc:3037] T b8e4c1bf9e7347cab50aad4c7d31182f P 6e0e6820452e421b9294f8a0de94b2b6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:07.593917 18436 raft_consensus.cc:3037] T b8e4c1bf9e7347cab50aad4c7d31182f P 67117d7b7c1d41b89eeed7d826143c1f [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:07.596311 18298 raft_consensus.cc:2454] T b8e4c1bf9e7347cab50aad4c7d31182f P 6e0e6820452e421b9294f8a0de94b2b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a90fa2f65624f8382dceb89d56cae23 in term 1.
I20260809 04:44:07.596356 18436 raft_consensus.cc:2454] T b8e4c1bf9e7347cab50aad4c7d31182f P 67117d7b7c1d41b89eeed7d826143c1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a90fa2f65624f8382dceb89d56cae23 in term 1.
I20260809 04:44:07.596722 18504 leader_election.cc:302] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a90fa2f65624f8382dceb89d56cae23, 67117d7b7c1d41b89eeed7d826143c1f; no voters:
I20260809 04:44:07.596971 18645 raft_consensus.cc:2781] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:07.597463 18645 raft_consensus.cc:686] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23 [term 1 LEADER]: Becoming Leader. State: Replica: 5a90fa2f65624f8382dceb89d56cae23, State: Running, Role: LEADER
I20260809 04:44:07.597872 18645 consensus_queue.cc:237] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0e6820452e421b9294f8a0de94b2b6" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 33017 } } peers { permanent_uuid: "5a90fa2f65624f8382dceb89d56cae23" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42559 } } peers { permanent_uuid: "67117d7b7c1d41b89eeed7d826143c1f" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42333 } }
I20260809 04:44:07.602120 18150 catalog_manager.cc:5495] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a90fa2f65624f8382dceb89d56cae23 (127.12.234.3). New cstate: current_term: 1 leader_uuid: "5a90fa2f65624f8382dceb89d56cae23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e0e6820452e421b9294f8a0de94b2b6" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a90fa2f65624f8382dceb89d56cae23" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "67117d7b7c1d41b89eeed7d826143c1f" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42333 } health_report { overall_health: UNKNOWN } } }
W20260809 04:44:07.638935 18621 tablet.cc:2369] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:44:07.643216 18635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:07.641176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35362 (local address 127.12.234.3:42559)
0809 04:44:07.641400 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:07.641405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:07.641423 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:44:07.641551 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:07.641555 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:07.641629 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:07.641731 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:07.642580 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:07.642585 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:07.642589 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:07.642608 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:07.642646 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:07.642662 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:07.642786 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:07.642890 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 04:44:07.642944 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260809 04:44:07.651265 18436 raft_consensus.cc:1261] T b8e4c1bf9e7347cab50aad4c7d31182f P 67117d7b7c1d41b89eeed7d826143c1f [term 1 FOLLOWER]: Refusing update from remote peer 5a90fa2f65624f8382dceb89d56cae23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:07.651264 18298 raft_consensus.cc:1261] T b8e4c1bf9e7347cab50aad4c7d31182f P 6e0e6820452e421b9294f8a0de94b2b6 [term 1 FOLLOWER]: Refusing update from remote peer 5a90fa2f65624f8382dceb89d56cae23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:07.652251 18648 consensus_queue.cc:1035] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67117d7b7c1d41b89eeed7d826143c1f" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:07.652631 18645 consensus_queue.cc:1035] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e0e6820452e421b9294f8a0de94b2b6" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:07.677107 18150 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:07.677407 18150 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:07.681043 18150 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8fed6469b3484b80b89d13b0598c88bb: Sending DeleteTablet for 3 replicas of tablet b8e4c1bf9e7347cab50aad4c7d31182f
I20260809 04:44:07.682076 18276 tablet_service.cc:1514] Processing DeleteTablet for tablet b8e4c1bf9e7347cab50aad4c7d31182f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:07 UTC) from {username='slave'} at 127.0.0.1:36006
I20260809 04:44:07.682516 18552 tablet_service.cc:1514] Processing DeleteTablet for tablet b8e4c1bf9e7347cab50aad4c7d31182f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:07 UTC) from {username='slave'} at 127.0.0.1:35352
I20260809 04:44:07.682799 18416 tablet_service.cc:1514] Processing DeleteTablet for tablet b8e4c1bf9e7347cab50aad4c7d31182f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:07 UTC) from {username='slave'} at 127.0.0.1:34144
I20260809 04:44:07.683643 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 18212
I20260809 04:44:07.698129 18663 tablet_replica.cc:330] T b8e4c1bf9e7347cab50aad4c7d31182f P 67117d7b7c1d41b89eeed7d826143c1f: stopping tablet replica
I20260809 04:44:07.698665 18663 raft_consensus.cc:2229] T b8e4c1bf9e7347cab50aad4c7d31182f P 67117d7b7c1d41b89eeed7d826143c1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:44:07.699309 18663 raft_consensus.cc:2258] T b8e4c1bf9e7347cab50aad4c7d31182f P 67117d7b7c1d41b89eeed7d826143c1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:07.700907 18633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:07.685336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36022 (local address 127.12.234.1:33017)
0809 04:44:07.685540 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:07.685548 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:07.695521 (+ 9973us) server_negotiation.cc:407] Connection header received
0809 04:44:07.696029 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:07.696039 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:07.696200 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:07.696288 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:07.699760 (+ 3472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:07.699770 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:07.699778 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:07.699807 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:07.699858 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:07.699892 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:07.700015 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:07.700341 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 04:44:07.700422 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 04:44:07.702099 18340 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:07.706279 18662 tablet_replica.cc:330] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23: stopping tablet replica
I20260809 04:44:07.707592 18662 raft_consensus.cc:2229] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:44:07.708130 18663 ts_tablet_manager.cc:1905] T b8e4c1bf9e7347cab50aad4c7d31182f P 67117d7b7c1d41b89eeed7d826143c1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:07.709940 18662 raft_consensus.cc:2258] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:07.712600 18662 ts_tablet_manager.cc:1905] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:07.716253 18663 ts_tablet_manager.cc:1918] T b8e4c1bf9e7347cab50aad4c7d31182f P 67117d7b7c1d41b89eeed7d826143c1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:07.716414 18663 log.cc:1196] T b8e4c1bf9e7347cab50aad4c7d31182f P 67117d7b7c1d41b89eeed7d826143c1f: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-1/wal/wals/b8e4c1bf9e7347cab50aad4c7d31182f
I20260809 04:44:07.716962 18663 ts_tablet_manager.cc:1939] T b8e4c1bf9e7347cab50aad4c7d31182f P 67117d7b7c1d41b89eeed7d826143c1f: Deleting consensus metadata
I20260809 04:44:07.718869 18136 catalog_manager.cc:4841] TS 67117d7b7c1d41b89eeed7d826143c1f (127.12.234.2:42333): tablet b8e4c1bf9e7347cab50aad4c7d31182f (table test-table [id=f4963378ff98492882326af9a7ef4ca3]) successfully deleted
I20260809 04:44:07.721017 18662 ts_tablet_manager.cc:1918] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:07.721215 18662 log.cc:1196] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786250603326224-13224-0/minicluster-data/ts-2/wal/wals/b8e4c1bf9e7347cab50aad4c7d31182f
I20260809 04:44:07.721863 18662 ts_tablet_manager.cc:1939] T b8e4c1bf9e7347cab50aad4c7d31182f P 5a90fa2f65624f8382dceb89d56cae23: Deleting consensus metadata
I20260809 04:44:07.723726 18136 catalog_manager.cc:4841] TS 5a90fa2f65624f8382dceb89d56cae23 (127.12.234.3:42559): tablet b8e4c1bf9e7347cab50aad4c7d31182f (table test-table [id=f4963378ff98492882326af9a7ef4ca3]) successfully deleted
I20260809 04:44:07.801862 18661 tablet_replica.cc:330] T b8e4c1bf9e7347cab50aad4c7d31182f P 6e0e6820452e421b9294f8a0de94b2b6: stopping tablet replica
I20260809 04:44:07.802277 18661 raft_consensus.cc:2229] T b8e4c1bf9e7347cab50aad4c7d31182f P 6e0e6820452e421b9294f8a0de94b2b6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:44:07.802846 18661 raft_consensus.cc:2258] T b8e4c1bf9e7347cab50aad4c7d31182f P 6e0e6820452e421b9294f8a0de94b2b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:07.803198 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 18212
I20260809 04:44:07.829366 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 18349
W20260809 04:44:07.829398 18138 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.234.1:33017 (error 108)
I20260809 04:44:07.832942 18637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:07.830548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34148 (local address 127.12.234.2:42333)
0809 04:44:07.830785 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:07.830794 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:07.830819 (+ 25us) server_negotiation.cc:407] Connection header received
0809 04:44:07.830902 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:07.830910 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:07.831088 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:07.831231 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:07.832205 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:07.832210 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:07.832214 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:07.832232 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:07.832269 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:07.832287 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:07.832402 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:07.832550 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 04:44:07.832610 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 04:44:07.833215 18634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:07.832663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:33017 (local address 127.0.0.1:36038)
0809 04:44:07.832908 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:07.833045 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:33017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
W20260809 04:44:07.833459 18138 catalog_manager.cc:4584] TS 6e0e6820452e421b9294f8a0de94b2b6 (127.12.234.1:33017): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b8e4c1bf9e7347cab50aad4c7d31182f: Network error: Client connection negotiation failed: client connection to 127.12.234.1:33017: connect: Connection refused (error 111)
I20260809 04:44:07.833679 18478 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:07.887130 18634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:07.886540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:33017 (local address 127.0.0.1:36042)
0809 04:44:07.886814 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:07.886934 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:33017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 04:44:07.888285 18634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:07.887864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:33017 (local address 127.0.0.1:36048)
0809 04:44:07.888024 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:07.888139 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:33017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 04:44:07.922027 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 18349
I20260809 04:44:07.949424 18634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:07.948772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:33017 (local address 127.0.0.1:36050)
0809 04:44:07.949107 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:07.949218 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:33017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179}
I20260809 04:44:07.950722 18634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:07.950328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:33017 (local address 127.0.0.1:36052)
0809 04:44:07.950530 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:07.950605 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:33017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 04:44:07.954010 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 18487
I20260809 04:44:07.957108 18635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:07.954831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35376 (local address 127.12.234.3:42559)
0809 04:44:07.955048 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:07.955054 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:07.955075 (+ 21us) server_negotiation.cc:407] Connection header received
0809 04:44:07.955238 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:07.955242 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:07.955293 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:07.955407 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:07.956434 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:07.956440 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:07.956445 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:07.956463 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:07.956504 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:07.956515 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:07.956680 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:07.956753 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 04:44:07.956811 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 04:44:07.957818 18613 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:08.031808 18634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:08.031229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:33017 (local address 127.0.0.1:36058)
0809 04:44:08.031503 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:08.031615 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:33017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260809 04:44:08.033141 18634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:08.032667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:33017 (local address 127.0.0.1:36064)
0809 04:44:08.032897 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:08.032986 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:33017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260809 04:44:08.082010 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 18487
I20260809 04:44:08.108143 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 18119
I20260809 04:44:08.109884 18182 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:08.215657 18634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:08.214899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:33017 (local address 127.0.0.1:36072)
0809 04:44:08.215084 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:08.215273 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:33017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 04:44:08.216231 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 18119
2026-08-09T04:44:08Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2816 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 04:44:08.262470 13224 mini_oidc.cc:152] Starting JWKS server
I20260809 04:44:08.265466 13224 webserver.cc:460] Webserver started at https://127.0.0.1:41799/ using document root <none> and password file <none>
I20260809 04:44:08.265838 13224 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 04:44:08.266510 13224 webserver.cc:460] Webserver started at http://127.0.0.1:45431/ using document root <none> and password file <none>
2026-08-09T04:44:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:08Z Disabled control of system clock
I20260809 04:44:08.299501 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:36561
--webserver_interface=127.12.234.62
--webserver_port=0
--builtin_ntp_servers=127.12.234.20:43523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:36561
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41799/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250603326224-13224-0/testchainca.cert with env {}
W20260809 04:44:08.649861 18677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:08.650190 18677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:08.650360 18677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:08.657346 18677 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:08.657487 18677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:08.657580 18677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:08.657682 18677 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:08.657756 18677 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 04:44:08.657842 18677 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 04:44:08.657918 18677 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41799/jwks.json
I20260809 04:44:08.667551 18677 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:43523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:36561
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:36561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41799/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250603326224-13224-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:08.668921 18677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:08.670186 18677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:08.681890 18682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:08.681919 18683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:08.681865 18685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:08.682650 18677 server_base.cc:1034] running on GCE node
I20260809 04:44:08.683359 18677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:08.684718 18677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:08.685909 18677 hybrid_clock.cc:648] HybridClock initialized: now 1786250648685863 us; error 35 us; skew 500 ppm
I20260809 04:44:08.686218 18677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:44:08.691349 18677 webserver.cc:460] Webserver started at http://127.12.234.62:46751/ using document root <none> and password file <none>
I20260809 04:44:08.691994 18677 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:08.692144 18677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:08.692481 18677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:08.695329 18677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "fe8832baa1d74b7cbecf4ea33e0e174d"
format_stamp: "Formatted at 2026-08-09 04:44:08 on dist-test-slave-1jjb"
server_key: "c43b17e177355cad4780d41c8844c975"
server_key_iv: "84b7896e1c8cc15dcb98a6e1f0dd9d29"
server_key_version: "encryptionkey@0"
I20260809 04:44:08.696067 18677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "fe8832baa1d74b7cbecf4ea33e0e174d"
format_stamp: "Formatted at 2026-08-09 04:44:08 on dist-test-slave-1jjb"
server_key: "c43b17e177355cad4780d41c8844c975"
server_key_iv: "84b7896e1c8cc15dcb98a6e1f0dd9d29"
server_key_version: "encryptionkey@0"
I20260809 04:44:08.701058 18677 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 04:44:08.704384 18691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:08.705756 18677 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 04:44:08.705904 18677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "fe8832baa1d74b7cbecf4ea33e0e174d"
format_stamp: "Formatted at 2026-08-09 04:44:08 on dist-test-slave-1jjb"
server_key: "c43b17e177355cad4780d41c8844c975"
server_key_iv: "84b7896e1c8cc15dcb98a6e1f0dd9d29"
server_key_version: "encryptionkey@0"
I20260809 04:44:08.706084 18677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:08.720876 18677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:08.721886 18677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:08.722265 18677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:08.741904 18677 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:36561
I20260809 04:44:08.741930 18744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:36561 every 8 connection(s)
I20260809 04:44:08.743396 18677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:08.747418 18745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:08.753844 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 18677
I20260809 04:44:08.754088 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:44:08.754894 13224 external_mini_cluster.cc:1428] Setting key ee113dcb5d1f76876daafe36a26ee35f
I20260809 04:44:08.759408 18745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe8832baa1d74b7cbecf4ea33e0e174d: Bootstrap starting.
I20260809 04:44:08.762708 18745 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe8832baa1d74b7cbecf4ea33e0e174d: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:08.763760 18745 log.cc:824] T 00000000000000000000000000000000 P fe8832baa1d74b7cbecf4ea33e0e174d: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:08.766803 18745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe8832baa1d74b7cbecf4ea33e0e174d: No bootstrap required, opened a new log
I20260809 04:44:08.769220 18748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:08.757056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33050 (local address 127.12.234.62:36561)
0809 04:44:08.758109 (+ 1053us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:08.758120 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:08.758164 (+ 44us) server_negotiation.cc:407] Connection header received
0809 04:44:08.760006 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:08.760030 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:08.760538 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:08.760989 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:08.761580 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:08.762936 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:08.764441 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:08.765157 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:08.765374 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:08.766693 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:08.766705 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:08.766726 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:08.767368 (+ 642us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:08.767449 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:08.767471 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:08.767611 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:08.767847 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 04:44:08.768219 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":387,"threads_started":1}
I20260809 04:44:08.773231 13224 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:44:08.775683 18745 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe8832baa1d74b7cbecf4ea33e0e174d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe8832baa1d74b7cbecf4ea33e0e174d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36561 } }
I20260809 04:44:08.776067 18745 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe8832baa1d74b7cbecf4ea33e0e174d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:08.776244 18745 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe8832baa1d74b7cbecf4ea33e0e174d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe8832baa1d74b7cbecf4ea33e0e174d, State: Initialized, Role: FOLLOWER
I20260809 04:44:08.776849 18745 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe8832baa1d74b7cbecf4ea33e0e174d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe8832baa1d74b7cbecf4ea33e0e174d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36561 } }
I20260809 04:44:08.777052 18745 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe8832baa1d74b7cbecf4ea33e0e174d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:08.777204 18745 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe8832baa1d74b7cbecf4ea33e0e174d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:08.777380 18745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe8832baa1d74b7cbecf4ea33e0e174d [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:08.779773 18745 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe8832baa1d74b7cbecf4ea33e0e174d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe8832baa1d74b7cbecf4ea33e0e174d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36561 } }
I20260809 04:44:08.780335 18745 leader_election.cc:302] T 00000000000000000000000000000000 P fe8832baa1d74b7cbecf4ea33e0e174d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe8832baa1d74b7cbecf4ea33e0e174d; no voters:
I20260809 04:44:08.780956 18745 leader_election.cc:288] T 00000000000000000000000000000000 P fe8832baa1d74b7cbecf4ea33e0e174d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:08.781117 18752 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe8832baa1d74b7cbecf4ea33e0e174d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:08.781836 18752 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe8832baa1d74b7cbecf4ea33e0e174d [term 1 LEADER]: Becoming Leader. State: Replica: fe8832baa1d74b7cbecf4ea33e0e174d, State: Running, Role: LEADER
I20260809 04:44:08.782349 18752 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe8832baa1d74b7cbecf4ea33e0e174d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe8832baa1d74b7cbecf4ea33e0e174d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36561 } }
I20260809 04:44:08.782998 18745 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe8832baa1d74b7cbecf4ea33e0e174d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:08.785578 18753 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe8832baa1d74b7cbecf4ea33e0e174d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe8832baa1d74b7cbecf4ea33e0e174d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe8832baa1d74b7cbecf4ea33e0e174d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36561 } } }
I20260809 04:44:08.785929 18754 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe8832baa1d74b7cbecf4ea33e0e174d [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe8832baa1d74b7cbecf4ea33e0e174d. Latest consensus state: current_term: 1 leader_uuid: "fe8832baa1d74b7cbecf4ea33e0e174d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe8832baa1d74b7cbecf4ea33e0e174d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36561 } } }
I20260809 04:44:08.786221 18753 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe8832baa1d74b7cbecf4ea33e0e174d [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:08.786294 18754 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe8832baa1d74b7cbecf4ea33e0e174d [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:08.787158 18758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:08.794250 18758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:08.804689 18758 catalog_manager.cc:1347] Generated new cluster ID: ae391daf794447feb7bc5b8af3cba8a3
I20260809 04:44:08.804837 18758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:08.819067 18758 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:08.820552 18758 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:08.827339 18758 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe8832baa1d74b7cbecf4ea33e0e174d: Generated new TSK 0
I20260809 04:44:08.828217 18758 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 04:44:08.866180 18778 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 04:44:08.866483 18778 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 04:44:08.867663 18748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:08.844342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33062 (local address 127.12.234.62:36561)
0809 04:44:08.844587 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:08.844596 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:08.845147 (+ 551us) server_negotiation.cc:407] Connection header received
0809 04:44:08.845535 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:08.845547 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:08.845643 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:08.845806 (+ 163us) server_negotiation.cc:226] Negotiated authn=JWT
0809 04:44:08.846343 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:08.847502 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:08.849045 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:08.849770 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:08.849921 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:08.851052 (+ 1131us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 04:44:08.866955 (+ 15903us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41799/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 04:44:08.867362 (+ 407us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33062: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41799/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 04:44:08.867789 18748 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33062: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41799/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 04:44:08.869515 18777 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:08.844308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:36561 (local address 127.0.0.1:33062)
0809 04:44:08.845003 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:08.845039 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:08.845242 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:08.845835 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:08.845839 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:08.845849 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0809 04:44:08.846168 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:08.846177 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:08.847670 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:08.847674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:08.848901 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:08.848913 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:08.850132 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:08.850156 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:08.850507 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:08.850626 (+ 119us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 04:44:08.867336 (+ 16710us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41799/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 04:44:08.867687 (+ 351us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.234.62:36561: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41799/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":356,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.12.234.62:36561: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41799/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 04:44:08.871358 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 18677
I20260809 04:44:08.872668 18740 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:08.969790 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 18677
2026-08-09T04:44:09Z chronyd exiting
I20260809 04:44:09.027974 13224 test_util.cc:175] -----------------------------------------------
I20260809 04:44:09.028179 13224 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786250603326224-13224-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (770 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 04:44:09.032003 13224 mini_oidc.cc:152] Starting JWKS server
I20260809 04:44:09.033869 13224 webserver.cc:460] Webserver started at https://127.0.0.1:42587/ using document root <none> and password file <none>
I20260809 04:44:09.034070 13224 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 04:44:09.034652 13224 webserver.cc:460] Webserver started at http://127.0.0.1:45427/ using document root <none> and password file <none>
2026-08-09T04:44:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:09Z Disabled control of system clock
I20260809 04:44:09.069280 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:37259
--webserver_interface=127.12.234.62
--webserver_port=0
--builtin_ntp_servers=127.12.234.20:45497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:37259
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42587/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250603326224-13224-0/testchainca.cert with env {}
W20260809 04:44:09.426229 18785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:09.426582 18785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:09.426748 18785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:09.434161 18785 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:09.434304 18785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:09.434404 18785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:09.434525 18785 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:09.434602 18785 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 04:44:09.434693 18785 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 04:44:09.434774 18785 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42587/jwks.json
I20260809 04:44:09.444891 18785 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:45497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:37259
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:37259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42587/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250603326224-13224-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:09.446285 18785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:09.447603 18785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:09.460043 18790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:09.460004 18791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:09.460021 18793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:09.460896 18785 server_base.cc:1034] running on GCE node
I20260809 04:44:09.461702 18785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:09.463142 18785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:09.464373 18785 hybrid_clock.cc:648] HybridClock initialized: now 1786250649464337 us; error 50 us; skew 500 ppm
I20260809 04:44:09.464748 18785 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:44:09.467083 18785 webserver.cc:460] Webserver started at http://127.12.234.62:46643/ using document root <none> and password file <none>
I20260809 04:44:09.467799 18785 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:09.467939 18785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:09.468346 18785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:09.472128 18785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "74e2a3b5e674432a819bb7c2a440750d"
format_stamp: "Formatted at 2026-08-09 04:44:09 on dist-test-slave-1jjb"
server_key: "b7e9b40a7687bfcc8c6c780b42a02c1a"
server_key_iv: "1568dde4ad36b0745c6bfb349f0278f2"
server_key_version: "encryptionkey@0"
I20260809 04:44:09.473387 18785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "74e2a3b5e674432a819bb7c2a440750d"
format_stamp: "Formatted at 2026-08-09 04:44:09 on dist-test-slave-1jjb"
server_key: "b7e9b40a7687bfcc8c6c780b42a02c1a"
server_key_iv: "1568dde4ad36b0745c6bfb349f0278f2"
server_key_version: "encryptionkey@0"
I20260809 04:44:09.479802 18785 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260809 04:44:09.483383 18799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:09.484752 18785 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:44:09.484946 18785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "74e2a3b5e674432a819bb7c2a440750d"
format_stamp: "Formatted at 2026-08-09 04:44:09 on dist-test-slave-1jjb"
server_key: "b7e9b40a7687bfcc8c6c780b42a02c1a"
server_key_iv: "1568dde4ad36b0745c6bfb349f0278f2"
server_key_version: "encryptionkey@0"
I20260809 04:44:09.485179 18785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:09.536278 18785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:09.537204 18785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:09.537544 18785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:09.557480 18785 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:37259
I20260809 04:44:09.557507 18851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:37259 every 8 connection(s)
I20260809 04:44:09.558898 18785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:09.562717 18852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:09.564759 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 18785
I20260809 04:44:09.564971 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:44:09.565814 13224 external_mini_cluster.cc:1428] Setting key 9dc39e205cad95e6a6465221688a0630
I20260809 04:44:09.575455 18852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74e2a3b5e674432a819bb7c2a440750d: Bootstrap starting.
I20260809 04:44:09.579077 18852 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74e2a3b5e674432a819bb7c2a440750d: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:09.579357 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:09.567799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34210 (local address 127.12.234.62:37259)
0809 04:44:09.568936 (+ 1137us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:09.568949 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:09.568989 (+ 40us) server_negotiation.cc:407] Connection header received
0809 04:44:09.570678 (+ 1689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:09.570698 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:09.571112 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:09.571433 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:09.572002 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:09.573352 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:09.575086 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:09.575790 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:09.575930 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:09.577148 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:09.577158 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:09.577172 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:09.577691 (+ 519us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:09.577761 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:09.577781 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:09.577872 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:09.578098 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 04:44:09.578415 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":709,"thread_start_us":492,"threads_started":1}
I20260809 04:44:09.580281 18852 log.cc:824] T 00000000000000000000000000000000 P 74e2a3b5e674432a819bb7c2a440750d: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:09.583422 18852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74e2a3b5e674432a819bb7c2a440750d: No bootstrap required, opened a new log
I20260809 04:44:09.583495 13224 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:44:09.591058 18852 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74e2a3b5e674432a819bb7c2a440750d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e2a3b5e674432a819bb7c2a440750d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 37259 } }
I20260809 04:44:09.591434 18852 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74e2a3b5e674432a819bb7c2a440750d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:09.591564 18852 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74e2a3b5e674432a819bb7c2a440750d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e2a3b5e674432a819bb7c2a440750d, State: Initialized, Role: FOLLOWER
I20260809 04:44:09.592145 18852 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74e2a3b5e674432a819bb7c2a440750d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e2a3b5e674432a819bb7c2a440750d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 37259 } }
I20260809 04:44:09.592432 18852 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74e2a3b5e674432a819bb7c2a440750d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:09.592629 18852 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74e2a3b5e674432a819bb7c2a440750d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:09.592803 18852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74e2a3b5e674432a819bb7c2a440750d [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:09.594940 18852 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74e2a3b5e674432a819bb7c2a440750d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e2a3b5e674432a819bb7c2a440750d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 37259 } }
I20260809 04:44:09.595428 18852 leader_election.cc:302] T 00000000000000000000000000000000 P 74e2a3b5e674432a819bb7c2a440750d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74e2a3b5e674432a819bb7c2a440750d; no voters:
I20260809 04:44:09.596149 18852 leader_election.cc:288] T 00000000000000000000000000000000 P 74e2a3b5e674432a819bb7c2a440750d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:09.596263 18857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74e2a3b5e674432a819bb7c2a440750d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:09.597128 18857 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74e2a3b5e674432a819bb7c2a440750d [term 1 LEADER]: Becoming Leader. State: Replica: 74e2a3b5e674432a819bb7c2a440750d, State: Running, Role: LEADER
I20260809 04:44:09.597638 18857 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74e2a3b5e674432a819bb7c2a440750d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e2a3b5e674432a819bb7c2a440750d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 37259 } }
I20260809 04:44:09.598325 18852 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74e2a3b5e674432a819bb7c2a440750d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:09.600570 18859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e2a3b5e674432a819bb7c2a440750d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74e2a3b5e674432a819bb7c2a440750d. Latest consensus state: current_term: 1 leader_uuid: "74e2a3b5e674432a819bb7c2a440750d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e2a3b5e674432a819bb7c2a440750d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 37259 } } }
I20260809 04:44:09.601099 18859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e2a3b5e674432a819bb7c2a440750d [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:09.601348 18858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e2a3b5e674432a819bb7c2a440750d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74e2a3b5e674432a819bb7c2a440750d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e2a3b5e674432a819bb7c2a440750d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 37259 } } }
I20260809 04:44:09.601751 18858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e2a3b5e674432a819bb7c2a440750d [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:09.603425 18863 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:09.608572 18863 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:09.617504 18863 catalog_manager.cc:1347] Generated new cluster ID: 5f1f847366c149899c3ba456bcc4c59d
I20260809 04:44:09.617625 18863 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:09.635102 18863 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:09.636534 18863 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:09.644162 18863 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74e2a3b5e674432a819bb7c2a440750d: Generated new TSK 0
I20260809 04:44:09.644955 18863 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 04:44:09.687312 18883 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 04:44:09.687471 18883 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 04:44:09.688534 18855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:09.661287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34224 (local address 127.12.234.62:37259)
0809 04:44:09.661586 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:09.661594 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:09.661920 (+ 326us) server_negotiation.cc:407] Connection header received
0809 04:44:09.662175 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:09.662179 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:09.662249 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:09.662391 (+ 142us) server_negotiation.cc:226] Negotiated authn=JWT
0809 04:44:09.662935 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:09.663634 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:09.665618 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:09.666071 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:09.666162 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:09.666765 (+ 603us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 04:44:09.687864 (+ 21099us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42587/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 04:44:09.688272 (+ 408us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34224: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42587/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":80}
W20260809 04:44:09.688692 18855 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34224: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42587/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 04:44:09.688890 18882 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:09.661175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:37259 (local address 127.0.0.1:34224)
0809 04:44:09.661809 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:09.661829 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:09.661985 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:09.662438 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:09.662445 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:09.662459 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0809 04:44:09.662770 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:09.662784 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:09.663771 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:09.663777 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:09.665463 (+ 1686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:09.665477 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:09.666235 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:09.666242 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:09.666465 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:09.666511 (+ 46us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 04:44:09.688146 (+ 21635us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42587/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 04:44:09.688482 (+ 336us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.234.62:37259: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42587/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":369,"threads_started":1}
I20260809 04:44:09.690184 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 18785
I20260809 04:44:09.691522 18847 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:09.784847 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 18785
2026-08-09T04:44:09Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (813 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 04:44:09.845697 13224 mini_oidc.cc:152] Starting JWKS server
I20260809 04:44:09.847733 13224 webserver.cc:460] Webserver started at https://127.0.0.1:45237/ using document root <none> and password file <none>
I20260809 04:44:09.848001 13224 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 04:44:09.848666 13224 webserver.cc:460] Webserver started at http://127.0.0.1:41863/ using document root <none> and password file <none>
2026-08-09T04:44:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:09Z Disabled control of system clock
I20260809 04:44:09.884867 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:42243
--webserver_interface=127.12.234.62
--webserver_port=0
--builtin_ntp_servers=127.12.234.20:39577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:42243
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45237/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 04:44:10.197644 18890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:10.197947 18890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:10.198100 18890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:10.204382 18890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:10.204501 18890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:10.204589 18890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:10.204701 18890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:10.204782 18890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 04:44:10.204862 18890 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 04:44:10.204917 18890 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45237/jwks.json
I20260809 04:44:10.213312 18890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:39577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:42243
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:42243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45237/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:10.214520 18890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:10.216238 18890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:10.227591 18896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:10.227555 18897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:10.228063 18899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:10.228577 18890 server_base.cc:1034] running on GCE node
I20260809 04:44:10.229308 18890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:10.230573 18890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:10.231758 18890 hybrid_clock.cc:648] HybridClock initialized: now 1786250650231714 us; error 45 us; skew 500 ppm
I20260809 04:44:10.232132 18890 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:44:10.234316 18890 webserver.cc:460] Webserver started at http://127.12.234.62:44495/ using document root <none> and password file <none>
I20260809 04:44:10.234961 18890 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:10.235090 18890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:10.235402 18890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:10.238448 18890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "7a264d224707474dacab1da8bf091e9f"
format_stamp: "Formatted at 2026-08-09 04:44:10 on dist-test-slave-1jjb"
server_key: "35f91802a04021c4309d923d62569c44"
server_key_iv: "ebc980466b6686b433c14dec9946fa7a"
server_key_version: "encryptionkey@0"
I20260809 04:44:10.239249 18890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "7a264d224707474dacab1da8bf091e9f"
format_stamp: "Formatted at 2026-08-09 04:44:10 on dist-test-slave-1jjb"
server_key: "35f91802a04021c4309d923d62569c44"
server_key_iv: "ebc980466b6686b433c14dec9946fa7a"
server_key_version: "encryptionkey@0"
I20260809 04:44:10.244197 18890 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260809 04:44:10.247403 18905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:10.248658 18890 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260809 04:44:10.248865 18890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "7a264d224707474dacab1da8bf091e9f"
format_stamp: "Formatted at 2026-08-09 04:44:10 on dist-test-slave-1jjb"
server_key: "35f91802a04021c4309d923d62569c44"
server_key_iv: "ebc980466b6686b433c14dec9946fa7a"
server_key_version: "encryptionkey@0"
I20260809 04:44:10.249118 18890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:10.261478 18890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:10.262195 18890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:10.262495 18890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:10.280161 18890 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:42243
I20260809 04:44:10.280241 18959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:42243 every 8 connection(s)
I20260809 04:44:10.281368 18890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:10.284508 18960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:10.287714 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 18890
I20260809 04:44:10.287955 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:44:10.288753 13224 external_mini_cluster.cc:1428] Setting key 1fd332288a6a0bee1ab7b817487cb66e
I20260809 04:44:10.295928 18960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a264d224707474dacab1da8bf091e9f: Bootstrap starting.
I20260809 04:44:10.298790 18960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a264d224707474dacab1da8bf091e9f: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:10.299790 18960 log.cc:824] T 00000000000000000000000000000000 P 7a264d224707474dacab1da8bf091e9f: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:10.301020 18963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:10.290525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35080 (local address 127.12.234.62:42243)
0809 04:44:10.291330 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:10.291338 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:10.291377 (+ 39us) server_negotiation.cc:407] Connection header received
0809 04:44:10.292943 (+ 1566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:10.292964 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:10.293380 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:10.293756 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:10.294282 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:10.295328 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:10.296727 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:10.297394 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:10.297596 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:10.298714 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:10.298721 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:10.298734 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:10.299293 (+ 559us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:10.299378 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:10.299398 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:10.299533 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:10.299761 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 04:44:10.300143 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":285,"threads_started":1}
I20260809 04:44:10.302610 18960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a264d224707474dacab1da8bf091e9f: No bootstrap required, opened a new log
I20260809 04:44:10.304582 13224 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:44:10.310261 18960 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a264d224707474dacab1da8bf091e9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a264d224707474dacab1da8bf091e9f" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42243 } }
I20260809 04:44:10.310583 18960 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a264d224707474dacab1da8bf091e9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:10.310694 18960 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a264d224707474dacab1da8bf091e9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a264d224707474dacab1da8bf091e9f, State: Initialized, Role: FOLLOWER
I20260809 04:44:10.311297 18960 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a264d224707474dacab1da8bf091e9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a264d224707474dacab1da8bf091e9f" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42243 } }
I20260809 04:44:10.311558 18960 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a264d224707474dacab1da8bf091e9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:10.311720 18960 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a264d224707474dacab1da8bf091e9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:10.311888 18960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a264d224707474dacab1da8bf091e9f [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:10.313939 18960 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a264d224707474dacab1da8bf091e9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a264d224707474dacab1da8bf091e9f" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42243 } }
I20260809 04:44:10.314402 18960 leader_election.cc:302] T 00000000000000000000000000000000 P 7a264d224707474dacab1da8bf091e9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a264d224707474dacab1da8bf091e9f; no voters:
I20260809 04:44:10.315004 18960 leader_election.cc:288] T 00000000000000000000000000000000 P 7a264d224707474dacab1da8bf091e9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:10.315197 18965 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a264d224707474dacab1da8bf091e9f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:10.315881 18965 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a264d224707474dacab1da8bf091e9f [term 1 LEADER]: Becoming Leader. State: Replica: 7a264d224707474dacab1da8bf091e9f, State: Running, Role: LEADER
I20260809 04:44:10.316491 18965 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a264d224707474dacab1da8bf091e9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a264d224707474dacab1da8bf091e9f" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42243 } }
I20260809 04:44:10.317250 18960 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a264d224707474dacab1da8bf091e9f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:10.319448 18967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a264d224707474dacab1da8bf091e9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a264d224707474dacab1da8bf091e9f. Latest consensus state: current_term: 1 leader_uuid: "7a264d224707474dacab1da8bf091e9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a264d224707474dacab1da8bf091e9f" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42243 } } }
I20260809 04:44:10.319403 18966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a264d224707474dacab1da8bf091e9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a264d224707474dacab1da8bf091e9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a264d224707474dacab1da8bf091e9f" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42243 } } }
I20260809 04:44:10.319801 18967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a264d224707474dacab1da8bf091e9f [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:10.319871 18966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a264d224707474dacab1da8bf091e9f [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:10.321200 18974 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:10.326632 18974 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:10.334615 18974 catalog_manager.cc:1347] Generated new cluster ID: df989fbe148b4f66bbd45d7d6da78711
I20260809 04:44:10.334724 18974 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:10.342928 18974 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:10.344007 18974 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:10.350052 18974 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a264d224707474dacab1da8bf091e9f: Generated new TSK 0
I20260809 04:44:10.350762 18974 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:44:10.370975 18963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:10.363242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35092 (local address 127.12.234.62:42243)
0809 04:44:10.363409 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:10.363414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:10.363752 (+ 338us) server_negotiation.cc:407] Connection header received
0809 04:44:10.363925 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:10.363930 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:10.364018 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:10.364179 (+ 161us) server_negotiation.cc:226] Negotiated authn=JWT
0809 04:44:10.364561 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:10.365597 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:10.366946 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:10.367548 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:10.367661 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:10.368095 (+ 434us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 04:44:10.370376 (+ 2281us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45237/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 04:44:10.370697 (+ 321us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35092: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45237/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 04:44:10.371104 18963 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35092: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45237/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 04:44:10.372571 18991 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:10.363104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:42243 (local address 127.0.0.1:35092)
0809 04:44:10.363664 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:10.363677 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:10.363778 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:10.364206 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:10.364209 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:10.364218 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0809 04:44:10.364428 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:10.364435 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:10.365755 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:10.365759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:10.366799 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:10.366810 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:10.367713 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:10.367717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:10.367830 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:10.367856 (+ 26us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 04:44:10.370596 (+ 2740us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45237/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 04:44:10.372204 (+ 1608us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.234.62:42243: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45237/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":1305,"thread_start_us":301,"threads_started":1}
I20260809 04:44:10.373544 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 18890
I20260809 04:44:10.374523 18955 generic_service.cc:196] Checking for leaks (request via RPC)
E20260809 04:44:10.382050 18992 webserver.cc:533] Webserver: sslize failed:
E20260809 04:44:10.382148 18992 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 04:44:10.455267 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 18890
2026-08-09T04:44:10Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (666 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T04:44:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:10Z Disabled control of system clock
I20260809 04:44:10.541095 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:46603
--webserver_interface=127.12.234.62
--webserver_port=0
--builtin_ntp_servers=127.12.234.20:33101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:46603
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250603326224-13224-0/testchainca.cert with env {}
W20260809 04:44:10.834232 18998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:10.834470 18998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:10.834550 18998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:10.840888 18998 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:10.840965 18998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:10.841049 18998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:10.841135 18998 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:10.841207 18998 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 04:44:10.841301 18998 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 04:44:10.841367 18998 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 04:44:10.849792 18998 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:33101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:46603
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:46603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250603326224-13224-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:10.850950 18998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:10.852135 18998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:10.863102 19003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:10.863134 19004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:10.863320 19006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:10.863680 18998 server_base.cc:1034] running on GCE node
I20260809 04:44:10.864519 18998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:10.865782 18998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:10.867027 18998 hybrid_clock.cc:648] HybridClock initialized: now 1786250650866968 us; error 54 us; skew 500 ppm
I20260809 04:44:10.867379 18998 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:44:10.869436 18998 webserver.cc:460] Webserver started at http://127.12.234.62:35575/ using document root <none> and password file <none>
I20260809 04:44:10.870081 18998 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:10.870196 18998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:10.870505 18998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:10.872932 18998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "ae2cd9850e8f4fd9b12f9e9578531b2d"
format_stamp: "Formatted at 2026-08-09 04:44:10 on dist-test-slave-1jjb"
server_key: "7ad425e45c7be11afc9e0b9ff2728980"
server_key_iv: "c0d3807eb05401a194cef8e113e1d244"
server_key_version: "encryptionkey@0"
I20260809 04:44:10.873654 18998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "ae2cd9850e8f4fd9b12f9e9578531b2d"
format_stamp: "Formatted at 2026-08-09 04:44:10 on dist-test-slave-1jjb"
server_key: "7ad425e45c7be11afc9e0b9ff2728980"
server_key_iv: "c0d3807eb05401a194cef8e113e1d244"
server_key_version: "encryptionkey@0"
I20260809 04:44:10.878278 18998 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:44:10.881089 19012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:10.882109 18998 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:44:10.882274 18998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "ae2cd9850e8f4fd9b12f9e9578531b2d"
format_stamp: "Formatted at 2026-08-09 04:44:10 on dist-test-slave-1jjb"
server_key: "7ad425e45c7be11afc9e0b9ff2728980"
server_key_iv: "c0d3807eb05401a194cef8e113e1d244"
server_key_version: "encryptionkey@0"
I20260809 04:44:10.882448 18998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:10.895586 18998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:10.896337 18998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:10.896607 18998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:10.912257 18998 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:46603
I20260809 04:44:10.912235 19064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:46603 every 8 connection(s)
I20260809 04:44:10.913301 18998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:10.916602 19065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:10.917590 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 18998
I20260809 04:44:10.917764 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:44:10.918321 13224 external_mini_cluster.cc:1428] Setting key 50fe0fce7651cb30d6b421b5d858a3aa
I20260809 04:44:10.929477 19065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae2cd9850e8f4fd9b12f9e9578531b2d: Bootstrap starting.
I20260809 04:44:10.931481 19068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:10.919973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53086 (local address 127.12.234.62:46603)
0809 04:44:10.921080 (+ 1107us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:10.921094 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:10.921136 (+ 42us) server_negotiation.cc:407] Connection header received
0809 04:44:10.922583 (+ 1447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:10.922610 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:10.923116 (+ 506us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:10.923480 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:10.923901 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:10.925586 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:10.926813 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:10.927772 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:10.927918 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:10.928901 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:10.928912 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:10.928926 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:10.929597 (+ 671us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:10.929689 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:10.929718 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:10.929840 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:10.930028 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 04:44:10.930317 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":316,"threads_started":1}
I20260809 04:44:10.933269 19065 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae2cd9850e8f4fd9b12f9e9578531b2d: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:10.934429 19065 log.cc:824] T 00000000000000000000000000000000 P ae2cd9850e8f4fd9b12f9e9578531b2d: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:10.935169 13224 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:44:10.937319 19065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae2cd9850e8f4fd9b12f9e9578531b2d: No bootstrap required, opened a new log
I20260809 04:44:10.943714 19065 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae2cd9850e8f4fd9b12f9e9578531b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae2cd9850e8f4fd9b12f9e9578531b2d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46603 } }
I20260809 04:44:10.943974 19065 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae2cd9850e8f4fd9b12f9e9578531b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:10.944036 19065 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae2cd9850e8f4fd9b12f9e9578531b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae2cd9850e8f4fd9b12f9e9578531b2d, State: Initialized, Role: FOLLOWER
I20260809 04:44:10.944522 19065 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae2cd9850e8f4fd9b12f9e9578531b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae2cd9850e8f4fd9b12f9e9578531b2d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46603 } }
I20260809 04:44:10.944702 19065 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae2cd9850e8f4fd9b12f9e9578531b2d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:10.944783 19065 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae2cd9850e8f4fd9b12f9e9578531b2d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:10.944923 19065 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae2cd9850e8f4fd9b12f9e9578531b2d [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:10.947033 19065 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae2cd9850e8f4fd9b12f9e9578531b2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae2cd9850e8f4fd9b12f9e9578531b2d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46603 } }
I20260809 04:44:10.947404 19065 leader_election.cc:302] T 00000000000000000000000000000000 P ae2cd9850e8f4fd9b12f9e9578531b2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae2cd9850e8f4fd9b12f9e9578531b2d; no voters:
I20260809 04:44:10.947866 19065 leader_election.cc:288] T 00000000000000000000000000000000 P ae2cd9850e8f4fd9b12f9e9578531b2d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:10.947949 19071 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae2cd9850e8f4fd9b12f9e9578531b2d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:10.948611 19071 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae2cd9850e8f4fd9b12f9e9578531b2d [term 1 LEADER]: Becoming Leader. State: Replica: ae2cd9850e8f4fd9b12f9e9578531b2d, State: Running, Role: LEADER
I20260809 04:44:10.949132 19071 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae2cd9850e8f4fd9b12f9e9578531b2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae2cd9850e8f4fd9b12f9e9578531b2d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46603 } }
I20260809 04:44:10.949904 19065 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae2cd9850e8f4fd9b12f9e9578531b2d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:10.951964 19074 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae2cd9850e8f4fd9b12f9e9578531b2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae2cd9850e8f4fd9b12f9e9578531b2d. Latest consensus state: current_term: 1 leader_uuid: "ae2cd9850e8f4fd9b12f9e9578531b2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae2cd9850e8f4fd9b12f9e9578531b2d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46603 } } }
I20260809 04:44:10.951939 19072 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae2cd9850e8f4fd9b12f9e9578531b2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae2cd9850e8f4fd9b12f9e9578531b2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae2cd9850e8f4fd9b12f9e9578531b2d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46603 } } }
I20260809 04:44:10.952320 19074 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae2cd9850e8f4fd9b12f9e9578531b2d [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:10.952385 19072 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae2cd9850e8f4fd9b12f9e9578531b2d [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:10.953675 19077 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:10.958415 19077 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:10.966791 19077 catalog_manager.cc:1347] Generated new cluster ID: f1d448534e374171b05524a636ccfcfa
I20260809 04:44:10.967000 19077 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:10.980412 19077 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:10.981400 19077 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:10.992331 19077 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae2cd9850e8f4fd9b12f9e9578531b2d: Generated new TSK 0
I20260809 04:44:10.992985 19077 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:44:11.437332 19068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:11.009858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53102 (local address 127.12.234.62:46603)
0809 04:44:11.010079 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:11.010089 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:11.010442 (+ 353us) server_negotiation.cc:407] Connection header received
0809 04:44:11.010630 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:11.010633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:11.010689 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:11.010803 (+ 114us) server_negotiation.cc:226] Negotiated authn=JWT
0809 04:44:11.011259 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.011869 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.013638 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.014013 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.014093 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:11.014567 (+ 474us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 04:44:11.436785 (+422218us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 04:44:11.437113 (+ 328us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53102: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 04:44:11.437486 19068 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53102: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 04:44:11.437615 19097 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:11.009785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46603 (local address 127.0.0.1:53102)
0809 04:44:11.010347 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:11.010364 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:11.010489 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:11.010836 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:11.010841 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:11.010852 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0809 04:44:11.011124 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:11.011136 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.011993 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.011998 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:11.013507 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:11.013520 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:11.014158 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:11.014165 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:11.014315 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:11.014355 (+ 40us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 04:44:11.437055 (+422700us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 04:44:11.437296 (+ 241us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.234.62:46603: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":296,"threads_started":1}
I20260809 04:44:11.438795 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 18998
I20260809 04:44:11.439863 19060 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:11.538964 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 18998
2026-08-09T04:44:11Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1066 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (377 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (375 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:12 dist-test-slave-1jjb krb5kdc[19112](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:12 dist-test-slave-1jjb krb5kdc[19112](info): set up 2 sockets
Aug 09 04:44:12 dist-test-slave-1jjb krb5kdc[19112](info): commencing operation
krb5kdc: starting...
W20260809 04:44:13.331485 13224 mini_kdc.cc:121] Time spent starting KDC: real 0.999s user 0.004s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:44:13 dist-test-slave-1jjb krb5kdc[19112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:44:13.531143 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:35039
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:42781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:35039
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:13.831115 19125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:13.831434 19125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:13.831526 19125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:13.838563 19125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:13.838649 19125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:13.838691 19125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:13.838737 19125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:13.838769 19125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:13.847105 19125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:42781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:35039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:35039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:13.848232 19125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:13.849402 19125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:13.860430 19134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:13.860450 19132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:13.860517 19131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:13.861028 19125 server_base.cc:1034] running on GCE node
I20260809 04:44:13.861732 19125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:13.862886 19125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:13.864161 19125 hybrid_clock.cc:648] HybridClock initialized: now 1786250653864094 us; error 44 us; skew 500 ppm
I20260809 04:44:13.864459 19125 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:13 dist-test-slave-1jjb krb5kdc[19112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250653, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:13.871666 19125 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:44:13.873034 19125 webserver.cc:460] Webserver started at http://127.12.234.62:44381/ using document root <none> and password file <none>
I20260809 04:44:13.873633 19125 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:13.873764 19125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:13.874055 19125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:13.876776 19125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "58c5869266264d7d8b80f5edaf452db9"
format_stamp: "Formatted at 2026-08-09 04:44:13 on dist-test-slave-1jjb"
server_key: "2fbf38bb04d7c15af5849e13816e088d"
server_key_iv: "400751b689f7d39d435057999702d25e"
server_key_version: "encryptionkey@0"
I20260809 04:44:13.877512 19125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "58c5869266264d7d8b80f5edaf452db9"
format_stamp: "Formatted at 2026-08-09 04:44:13 on dist-test-slave-1jjb"
server_key: "2fbf38bb04d7c15af5849e13816e088d"
server_key_iv: "400751b689f7d39d435057999702d25e"
server_key_version: "encryptionkey@0"
I20260809 04:44:13.882159 19125 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:44:13.885121 19141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:13.886289 19125 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:44:13.886478 19125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "58c5869266264d7d8b80f5edaf452db9"
format_stamp: "Formatted at 2026-08-09 04:44:13 on dist-test-slave-1jjb"
server_key: "2fbf38bb04d7c15af5849e13816e088d"
server_key_iv: "400751b689f7d39d435057999702d25e"
server_key_version: "encryptionkey@0"
I20260809 04:44:13.886682 19125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:13.915746 19125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:13.919191 19125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:13.919438 19125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:13.936306 19125 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:35039
I20260809 04:44:13.936379 19196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:35039 every 8 connection(s)
I20260809 04:44:13.937564 19125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:13.938184 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 19125
I20260809 04:44:13.938419 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:44:13.939095 13224 external_mini_cluster.cc:1428] Setting key 059512912efdeb70dfaeb439ab4422a7
I20260809 04:44:13.941454 19197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 04:44:13 dist-test-slave-1jjb krb5kdc[19112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:44:13.952904 19197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58c5869266264d7d8b80f5edaf452db9: Bootstrap starting.
I20260809 04:44:13.956815 19197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58c5869266264d7d8b80f5edaf452db9: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:13.957996 19197 log.cc:824] T 00000000000000000000000000000000 P 58c5869266264d7d8b80f5edaf452db9: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:13.961006 19197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58c5869266264d7d8b80f5edaf452db9: No bootstrap required, opened a new log
I20260809 04:44:13.961550 19200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:13.940876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46916 (local address 127.12.234.62:35039)
0809 04:44:13.941595 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:13.941607 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:13.941650 (+ 43us) server_negotiation.cc:407] Connection header received
0809 04:44:13.943680 (+ 2030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:13.943703 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:13.944135 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:13.944510 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:13.945605 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:13.947578 (+ 1973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:13.948752 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:13.949706 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:13.949881 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:13.953675 (+ 3794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:13.953685 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:13.953701 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:13.953740 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:13.957626 (+ 3886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:13.958209 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:13.958216 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:13.958223 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:13.958415 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:13.958851 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:13.958854 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:13.958856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:13.959407 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:13.959656 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:13.960018 (+ 362us) server_negotiation.cc:299] Negotiation successful
0809 04:44:13.960428 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":294,"threads_started":1}
I20260809 04:44:13.968312 19197 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58c5869266264d7d8b80f5edaf452db9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58c5869266264d7d8b80f5edaf452db9" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 35039 } }
I20260809 04:44:13.968607 19197 raft_consensus.cc:374] T 00000000000000000000000000000000 P 58c5869266264d7d8b80f5edaf452db9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:13.968674 19197 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58c5869266264d7d8b80f5edaf452db9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58c5869266264d7d8b80f5edaf452db9, State: Initialized, Role: FOLLOWER
I20260809 04:44:13.969172 19197 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58c5869266264d7d8b80f5edaf452db9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58c5869266264d7d8b80f5edaf452db9" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 35039 } }
I20260809 04:44:13.969362 19197 raft_consensus.cc:388] T 00000000000000000000000000000000 P 58c5869266264d7d8b80f5edaf452db9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:13.969455 19197 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58c5869266264d7d8b80f5edaf452db9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:13.969580 19197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58c5869266264d7d8b80f5edaf452db9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:13.971458 19197 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58c5869266264d7d8b80f5edaf452db9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58c5869266264d7d8b80f5edaf452db9" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 35039 } }
I20260809 04:44:13.971962 19197 leader_election.cc:302] T 00000000000000000000000000000000 P 58c5869266264d7d8b80f5edaf452db9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58c5869266264d7d8b80f5edaf452db9; no voters:
I20260809 04:44:13.972559 19197 leader_election.cc:288] T 00000000000000000000000000000000 P 58c5869266264d7d8b80f5edaf452db9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:13.972678 19202 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58c5869266264d7d8b80f5edaf452db9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:13.973341 19202 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58c5869266264d7d8b80f5edaf452db9 [term 1 LEADER]: Becoming Leader. State: Replica: 58c5869266264d7d8b80f5edaf452db9, State: Running, Role: LEADER
I20260809 04:44:13.973845 19202 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58c5869266264d7d8b80f5edaf452db9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58c5869266264d7d8b80f5edaf452db9" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 35039 } }
I20260809 04:44:13.974488 19197 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58c5869266264d7d8b80f5edaf452db9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:13.976879 19203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58c5869266264d7d8b80f5edaf452db9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58c5869266264d7d8b80f5edaf452db9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58c5869266264d7d8b80f5edaf452db9" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 35039 } } }
I20260809 04:44:13.976733 19204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58c5869266264d7d8b80f5edaf452db9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58c5869266264d7d8b80f5edaf452db9. Latest consensus state: current_term: 1 leader_uuid: "58c5869266264d7d8b80f5edaf452db9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58c5869266264d7d8b80f5edaf452db9" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 35039 } } }
I20260809 04:44:13.977334 19203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58c5869266264d7d8b80f5edaf452db9 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:13.977375 19204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58c5869266264d7d8b80f5edaf452db9 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:13.979802 19211 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:13.984887 19211 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:13.998050 19211 catalog_manager.cc:1347] Generated new cluster ID: c5fdcf55e75846aba1df392c2947a931
I20260809 04:44:13.998153 19211 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:14.024642 19211 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:14.025883 19211 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:14.036461 19211 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58c5869266264d7d8b80f5edaf452db9: Generated new TSK 0
I20260809 04:44:14.037228 19211 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
I20260809 04:44:14.093379 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:0
--local_ip_for_outbound_sockets=127.12.234.1
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:35039
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:42781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:14.409664 19224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:14.409904 19224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:14.410051 19224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:14.416538 19224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:14.416630 19224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:14.416832 19224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:44:14.425278 19224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:42781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:35039
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:14.426417 19224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:14.427599 19224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:14.439805 19229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:14.439836 19232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:14.439868 19230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:14.440912 19224 server_base.cc:1034] running on GCE node
I20260809 04:44:14.441395 19224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:14.442142 19224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:14.443347 19224 hybrid_clock.cc:648] HybridClock initialized: now 1786250654443309 us; error 50 us; skew 500 ppm
I20260809 04:44:14.443608 19224 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:14 dist-test-slave-1jjb krb5kdc[19112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250654, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:14.446343 19224 init.cc:377] Logged in from keytab as kudu/127.12.234.1@KRBTEST.COM (short username kudu)
I20260809 04:44:14.447666 19224 webserver.cc:460] Webserver started at http://127.12.234.1:42455/ using document root <none> and password file <none>
I20260809 04:44:14.448397 19224 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:14.448546 19224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:14.448843 19224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:14.451400 19224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-0/data/instance:
uuid: "8a70c945a8c741378e8efeea9ed97e2b"
format_stamp: "Formatted at 2026-08-09 04:44:14 on dist-test-slave-1jjb"
server_key: "313949c05118beec9d858f9322ea785f"
server_key_iv: "1cfe61e3b9ae63255c811f4c006749bd"
server_key_version: "encryptionkey@0"
I20260809 04:44:14.452164 19224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance:
uuid: "8a70c945a8c741378e8efeea9ed97e2b"
format_stamp: "Formatted at 2026-08-09 04:44:14 on dist-test-slave-1jjb"
server_key: "313949c05118beec9d858f9322ea785f"
server_key_iv: "1cfe61e3b9ae63255c811f4c006749bd"
server_key_version: "encryptionkey@0"
I20260809 04:44:14.456730 19224 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 04:44:14.459720 19240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:14.460933 19224 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:44:14.461090 19224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "8a70c945a8c741378e8efeea9ed97e2b"
format_stamp: "Formatted at 2026-08-09 04:44:14 on dist-test-slave-1jjb"
server_key: "313949c05118beec9d858f9322ea785f"
server_key_iv: "1cfe61e3b9ae63255c811f4c006749bd"
server_key_version: "encryptionkey@0"
I20260809 04:44:14.461270 19224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:14.476279 19224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:14.479858 19224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:14.480197 19224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:14.481093 19224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:14.482206 19224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:14.482268 19224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:14.482391 19224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:14.482445 19224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:14.513020 19224 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:33479
I20260809 04:44:14.513094 19357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:33479 every 8 connection(s)
I20260809 04:44:14.514183 19224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:44:14.523734 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 19224
I20260809 04:44:14.523972 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance
I20260809 04:44:14.524610 13224 external_mini_cluster.cc:1428] Setting key 1b1363ea7b3294c6b7afa5b908c05275
Aug 09 04:44:14 dist-test-slave-1jjb krb5kdc[19112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250654, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:44:14.533820 19360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:14.516494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:53029 (local address 127.12.234.62:35039)
0809 04:44:14.516942 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:14.516947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:14.517692 (+ 745us) server_negotiation.cc:407] Connection header received
0809 04:44:14.519216 (+ 1524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:14.519221 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:14.519318 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:14.519425 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:14.520746 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.521369 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.522687 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.523038 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.523199 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:14.527158 (+ 3959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:14.527166 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:14.527172 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:14.527223 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:14.530632 (+ 3409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:14.531497 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:14.531505 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:14.531511 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:14.531676 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:14.532055 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:14.532062 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:14.532067 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:14.532452 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:14.532598 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:14.533157 (+ 559us) server_negotiation.cc:299] Negotiation successful
0809 04:44:14.533370 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":252,"threads_started":1}
I20260809 04:44:14.534372 19362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:14.516871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:35039 (local address 127.12.234.1:53029)
0809 04:44:14.517538 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:14.517579 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:14.518977 (+ 1398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:14.519640 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:14.519646 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:14.520063 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:14.520591 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:14.520606 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.521519 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.521522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:14.522558 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:14.522569 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:14.523296 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:14.523305 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:14.523531 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:14.524226 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:14.524251 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:14.526931 (+ 2680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:14.530786 (+ 3855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:14.530796 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:14.530810 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:14.531309 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:14.531786 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:14.531789 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:14.531791 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:14.531954 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:14.532582 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:14.532587 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:14.532848 (+ 261us) client_negotiation.cc:769] Sending connection context
0809 04:44:14.533044 (+ 196us) client_negotiation.cc:240] Negotiation successful
0809 04:44:14.533356 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":301,"threads_started":1}
I20260809 04:44:14.535848 19358 heartbeater.cc:347] Connected to a master server at 127.12.234.62:35039
I20260809 04:44:14.536197 19358 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:14.536885 19358 heartbeater.cc:511] Master 127.12.234.62:35039 requested a full tablet report, sending...
I20260809 04:44:14.538894 19162 ts_manager.cc:194] Registered new tserver with Master: 8a70c945a8c741378e8efeea9ed97e2b (127.12.234.1:33479)
I20260809 04:44:14.540788 19162 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.1@KRBTEST.COM'} at 127.12.234.1:53029
WARNING: no policy specified for kudu/127.12.234.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
I20260809 04:44:14.572714 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:0
--local_ip_for_outbound_sockets=127.12.234.2
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:35039
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:42781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:14.864408 19365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:14.864660 19365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:14.864764 19365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:14.871392 19365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:14.871475 19365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:14.871573 19365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:44:14.880354 19365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:42781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:35039
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:14.881379 19365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:14.882454 19365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:14.894896 19371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:14.894857 19372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:14.894945 19374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:14.895550 19365 server_base.cc:1034] running on GCE node
I20260809 04:44:14.896154 19365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:14.897001 19365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:14.898229 19365 hybrid_clock.cc:648] HybridClock initialized: now 1786250654898181 us; error 60 us; skew 500 ppm
I20260809 04:44:14.898512 19365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:14 dist-test-slave-1jjb krb5kdc[19112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250654, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:14.901217 19365 init.cc:377] Logged in from keytab as kudu/127.12.234.2@KRBTEST.COM (short username kudu)
I20260809 04:44:14.902443 19365 webserver.cc:460] Webserver started at http://127.12.234.2:37625/ using document root <none> and password file <none>
I20260809 04:44:14.903095 19365 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:14.903257 19365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:14.903546 19365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:14.906337 19365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-1/data/instance:
uuid: "7af15948d5cb43b99e08bfe784337001"
format_stamp: "Formatted at 2026-08-09 04:44:14 on dist-test-slave-1jjb"
server_key: "b6d888bc85c0aaedfdd706c39552c860"
server_key_iv: "87d99d8ddd8e5dd8a1a9f2023ddf3fc3"
server_key_version: "encryptionkey@0"
I20260809 04:44:14.907112 19365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance:
uuid: "7af15948d5cb43b99e08bfe784337001"
format_stamp: "Formatted at 2026-08-09 04:44:14 on dist-test-slave-1jjb"
server_key: "b6d888bc85c0aaedfdd706c39552c860"
server_key_iv: "87d99d8ddd8e5dd8a1a9f2023ddf3fc3"
server_key_version: "encryptionkey@0"
I20260809 04:44:14.911614 19365 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260809 04:44:14.914623 19381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:14.915668 19365 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:44:14.915836 19365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "7af15948d5cb43b99e08bfe784337001"
format_stamp: "Formatted at 2026-08-09 04:44:14 on dist-test-slave-1jjb"
server_key: "b6d888bc85c0aaedfdd706c39552c860"
server_key_iv: "87d99d8ddd8e5dd8a1a9f2023ddf3fc3"
server_key_version: "encryptionkey@0"
I20260809 04:44:14.916023 19365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:14.968669 19365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:14.972339 19365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:14.972600 19365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:14.973423 19365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:14.974587 19365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:14.974681 19365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:14.974810 19365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:14.974874 19365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:15.005571 19365 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:42329
I20260809 04:44:15.005646 19499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:42329 every 8 connection(s)
I20260809 04:44:15.006907 19365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:44:15.010406 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 19365
I20260809 04:44:15.010591 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance
I20260809 04:44:15.011029 13224 external_mini_cluster.cc:1428] Setting key 9cf2a296afea80c7d7fd2ce9bf78e24a
Aug 09 04:44:15 dist-test-slave-1jjb krb5kdc[19112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250654, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:44:15.025389 19360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.009285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:51493 (local address 127.12.234.62:35039)
0809 04:44:15.009461 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:15.009466 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:15.010508 (+ 1042us) server_negotiation.cc:407] Connection header received
0809 04:44:15.012246 (+ 1738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:15.012252 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:15.012339 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:15.012441 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:15.013752 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.014419 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.015810 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.016290 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.016428 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.020220 (+ 3792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:15.020224 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:15.020228 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:15.020257 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:15.022674 (+ 2417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:15.023333 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:15.023342 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:15.023345 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:15.023462 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:15.023850 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:15.023855 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:15.023859 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:15.024144 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:15.024298 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:15.024871 (+ 573us) server_negotiation.cc:299] Negotiation successful
0809 04:44:15.025070 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 04:44:15.026281 19502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.009688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:35039 (local address 127.12.234.2:51493)
0809 04:44:15.010352 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:15.010394 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:15.011956 (+ 1562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:15.012663 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:15.012670 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:15.013105 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:15.013596 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:15.013612 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.014575 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.014579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:15.015662 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:15.015673 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.016515 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.016524 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:15.016756 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.017430 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:15.017450 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:15.019941 (+ 2491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:15.022796 (+ 2855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:15.022803 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:15.022815 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:15.023212 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:15.023575 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:15.023580 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:15.023583 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:15.023744 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:15.024266 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:15.024274 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:15.024548 (+ 274us) client_negotiation.cc:769] Sending connection context
0809 04:44:15.024797 (+ 249us) client_negotiation.cc:240] Negotiation successful
0809 04:44:15.025257 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":291,"threads_started":1}
I20260809 04:44:15.027626 19500 heartbeater.cc:347] Connected to a master server at 127.12.234.62:35039
I20260809 04:44:15.027981 19500 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:15.028785 19500 heartbeater.cc:511] Master 127.12.234.62:35039 requested a full tablet report, sending...
I20260809 04:44:15.030409 19162 ts_manager.cc:194] Registered new tserver with Master: 7af15948d5cb43b99e08bfe784337001 (127.12.234.2:42329)
I20260809 04:44:15.031587 19162 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.2@KRBTEST.COM'} at 127.12.234.2:51493
WARNING: no policy specified for kudu/127.12.234.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
I20260809 04:44:15.059306 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:0
--local_ip_for_outbound_sockets=127.12.234.3
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:35039
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:42781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:15.358947 19505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:15.359170 19505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:15.359269 19505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:15.365551 19505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:15.365665 19505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:15.365772 19505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:44:15.374109 19505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:42781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:35039
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:15.375124 19505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:15.376298 19505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:15.388164 19512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:15.388268 19511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:15.388244 19514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:15.389143 19505 server_base.cc:1034] running on GCE node
I20260809 04:44:15.389741 19505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:15.390628 19505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:15.391812 19505 hybrid_clock.cc:648] HybridClock initialized: now 1786250655391763 us; error 41 us; skew 500 ppm
I20260809 04:44:15.392159 19505 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:15 dist-test-slave-1jjb krb5kdc[19112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250655, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:15.395084 19505 init.cc:377] Logged in from keytab as kudu/127.12.234.3@KRBTEST.COM (short username kudu)
I20260809 04:44:15.396366 19505 webserver.cc:460] Webserver started at http://127.12.234.3:44531/ using document root <none> and password file <none>
I20260809 04:44:15.396950 19505 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:15.397058 19505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:15.397333 19505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:15.399684 19505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-2/data/instance:
uuid: "5f11de85bdd7419090ffb097abd1db1f"
format_stamp: "Formatted at 2026-08-09 04:44:15 on dist-test-slave-1jjb"
server_key: "e1c10bb9502ff34ef6c8668565813039"
server_key_iv: "d009930cfae03dd1d996bc6b3e8a6d7b"
server_key_version: "encryptionkey@0"
I20260809 04:44:15.400380 19505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance:
uuid: "5f11de85bdd7419090ffb097abd1db1f"
format_stamp: "Formatted at 2026-08-09 04:44:15 on dist-test-slave-1jjb"
server_key: "e1c10bb9502ff34ef6c8668565813039"
server_key_iv: "d009930cfae03dd1d996bc6b3e8a6d7b"
server_key_version: "encryptionkey@0"
I20260809 04:44:15.404876 19505 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260809 04:44:15.407881 19521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:15.409204 19505 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:44:15.409373 19505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "5f11de85bdd7419090ffb097abd1db1f"
format_stamp: "Formatted at 2026-08-09 04:44:15 on dist-test-slave-1jjb"
server_key: "e1c10bb9502ff34ef6c8668565813039"
server_key_iv: "d009930cfae03dd1d996bc6b3e8a6d7b"
server_key_version: "encryptionkey@0"
I20260809 04:44:15.409574 19505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:15.419216 19505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:15.422681 19505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:15.422931 19505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:15.423755 19505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:15.424935 19505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:15.425027 19505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:15.425176 19505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:15.425254 19505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:15.455506 19505 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:44037
I20260809 04:44:15.455567 19639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:44037 every 8 connection(s)
I20260809 04:44:15.456851 19505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:44:15.467128 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 19505
I20260809 04:44:15.467338 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance
I20260809 04:44:15.467751 13224 external_mini_cluster.cc:1428] Setting key cbeb21937a05d964dce24caf4fab1a13
Aug 09 04:44:15 dist-test-slave-1jjb krb5kdc[19112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250655, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:44:15.477584 19360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.459174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:53913 (local address 127.12.234.62:35039)
0809 04:44:15.459340 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:15.459344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:15.460605 (+ 1261us) server_negotiation.cc:407] Connection header received
0809 04:44:15.462252 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:15.462257 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:15.462338 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:15.462439 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:15.463703 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.464495 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.466015 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.466445 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.466590 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.470308 (+ 3718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:15.470314 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:15.470318 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:15.470353 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:15.473698 (+ 3345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:15.474609 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:15.474615 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:15.474619 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:15.474750 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:15.475247 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:15.475252 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:15.475255 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:15.475590 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:15.475753 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:15.476728 (+ 975us) server_negotiation.cc:299] Negotiation successful
0809 04:44:15.477070 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 04:44:15.478214 19642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.459637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:35039 (local address 127.12.234.3:53913)
0809 04:44:15.460430 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:15.460487 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:15.462010 (+ 1523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:15.462649 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:15.462655 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:15.463084 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:15.463549 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:15.463565 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.464646 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.464650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:15.465885 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:15.465897 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.466677 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.466686 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:15.466941 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.467599 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:15.467619 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:15.470122 (+ 2503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:15.473873 (+ 3751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:15.473881 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:15.473894 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:15.474490 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:15.474894 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:15.474898 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:15.474901 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:15.475142 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:15.475748 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:15.475755 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:15.476222 (+ 467us) client_negotiation.cc:769] Sending connection context
0809 04:44:15.476554 (+ 332us) client_negotiation.cc:240] Negotiation successful
0809 04:44:15.477055 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":386,"threads_started":1}
I20260809 04:44:15.479441 19640 heartbeater.cc:347] Connected to a master server at 127.12.234.62:35039
I20260809 04:44:15.479728 19640 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:15.480352 19640 heartbeater.cc:511] Master 127.12.234.62:35039 requested a full tablet report, sending...
I20260809 04:44:15.481747 19161 ts_manager.cc:194] Registered new tserver with Master: 5f11de85bdd7419090ffb097abd1db1f (127.12.234.3:44037)
I20260809 04:44:15.482586 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:44:15.482771 19161 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.3@KRBTEST.COM'} at 127.12.234.3:53913
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/test-admin.keytab.
Aug 09 04:44:15 dist-test-slave-1jjb krb5kdc[19112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:15.511197 13224 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 04:44:15.516218 13224 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/krb5cc' of type FILE
W20260809 04:44:15.516286 19647 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:15.513800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59508 (local address 127.12.234.1:33479)
0809 04:44:15.514488 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:15.514495 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:15.514525 (+ 30us) server_negotiation.cc:407] Connection header received
0809 04:44:15.514578 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:15.514583 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:15.514747 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:15.514869 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:15.515782 (+ 913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59508: BlockingRecv error: recv got EOF from 127.0.0.1:59508 (error 108)
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":432,"threads_started":1}
Aug 09 04:44:15 dist-test-slave-1jjb krb5kdc[19112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:15.517896 13224 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 04:44:15 dist-test-slave-1jjb krb5kdc[19112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:15.519125 13224 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 04:44:15 dist-test-slave-1jjb krb5kdc[19112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.1@KRBTEST.COM
I20260809 04:44:15.532702 19647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.520580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59516 (local address 127.12.234.1:33479)
0809 04:44:15.520736 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:15.520743 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:15.520866 (+ 123us) server_negotiation.cc:407] Connection header received
0809 04:44:15.520965 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:15.520968 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:15.521014 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:15.521089 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:15.521958 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.522859 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.523944 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.524463 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.524594 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.527711 (+ 3117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:15.527721 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:15.527732 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:15.527762 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:15.530314 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:15.530934 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:15.530939 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:15.530941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:15.531064 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:15.531382 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:15.531385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:15.531387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:15.531687 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:15.531875 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:15.532226 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 04:44:15.532486 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260809 04:44:15.537881 19647 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:15.536173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59530 (local address 127.12.234.1:33479)
0809 04:44:15.536385 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:15.536392 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:15.536642 (+ 250us) server_negotiation.cc:407] Connection header received
0809 04:44:15.536810 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:15.536815 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:15.536892 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:15.536992 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:15.537675 (+ 683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59530: BlockingRecv error: recv got EOF from 127.0.0.1:59530 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 04:44:15.538065 13224 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/krb5cc' of type FILE
Aug 09 04:44:15 dist-test-slave-1jjb krb5kdc[19112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:15.539292 13224 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 04:44:15 dist-test-slave-1jjb krb5kdc[19112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:15.540436 13224 init.cc:295] Successfully reacquired a new kerberos TGT
I20260809 04:44:15.543437 19358 heartbeater.cc:503] Master 127.12.234.62:35039 was elected leader, sending a full tablet report...
Aug 09 04:44:15 dist-test-slave-1jjb krb5kdc[19112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.1@KRBTEST.COM
I20260809 04:44:15.552621 19647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.541449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59536 (local address 127.12.234.1:33479)
0809 04:44:15.541573 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:15.541576 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:15.541878 (+ 302us) server_negotiation.cc:407] Connection header received
0809 04:44:15.541967 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:15.541969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:15.542010 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:15.542084 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:15.542862 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.543710 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.544889 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.545333 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.545425 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.548590 (+ 3165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:15.548594 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:15.548596 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:15.548620 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:15.550761 (+ 2141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:15.551255 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:15.551258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:15.551260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:15.551372 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:15.551654 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:15.551656 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:15.551658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:15.551868 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:15.552044 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:15.552314 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 04:44:15.552436 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260809 04:44:15.555857 19647 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:15.554680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59546 (local address 127.12.234.1:33479)
0809 04:44:15.554840 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:15.554845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:15.554959 (+ 114us) server_negotiation.cc:407] Connection header received
0809 04:44:15.555038 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:15.555040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:15.555091 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:15.555153 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:15.555677 (+ 524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59546: BlockingRecv error: recv got EOF from 127.0.0.1:59546 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 04:44:15.556325 13224 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786250603326224-13224-0/krb5kdc/krb5cc)
Aug 09 04:44:15 dist-test-slave-1jjb krb5kdc[19112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:15.557685 13224 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 04:44:15 dist-test-slave-1jjb krb5kdc[19112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:15.558753 13224 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 04:44:15 dist-test-slave-1jjb krb5kdc[19112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.1@KRBTEST.COM
I20260809 04:44:15.571750 19647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.559702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59548 (local address 127.12.234.1:33479)
0809 04:44:15.559868 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:15.559875 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:15.560125 (+ 250us) server_negotiation.cc:407] Connection header received
0809 04:44:15.560307 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:15.560310 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:15.560353 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:15.560457 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:15.561337 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.561932 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.562954 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.563445 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.563595 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.567533 (+ 3938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:15.567537 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:15.567540 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:15.567564 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:15.569892 (+ 2328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:15.570447 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:15.570450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:15.570452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:15.570546 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:15.570804 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:15.570807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:15.570808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:15.571009 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:15.571193 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:15.571336 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 04:44:15.571458 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 04:44:15.572978 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 19224
I20260809 04:44:15.582784 19647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.573537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59564 (local address 127.12.234.1:33479)
0809 04:44:15.573651 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:15.573655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:15.573966 (+ 311us) server_negotiation.cc:407] Connection header received
0809 04:44:15.574064 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:15.574066 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:15.574107 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:15.574169 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:15.574919 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.575440 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.576547 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.576954 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.577052 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.578639 (+ 1587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:15.578642 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:15.578645 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:15.578665 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:15.580912 (+ 2247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:15.581410 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:15.581413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:15.581415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:15.581525 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:15.581781 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:15.581784 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:15.581785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:15.581973 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:15.582131 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:15.582399 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 04:44:15.582563 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 04:44:15.583287 19353 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:15.656855 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 19224
I20260809 04:44:15.675426 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 19365
Aug 09 04:44:15 dist-test-slave-1jjb krb5kdc[19112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.2@KRBTEST.COM
I20260809 04:44:15.689494 19660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.676316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37086 (local address 127.12.234.2:42329)
0809 04:44:15.676794 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:15.676802 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:15.676827 (+ 25us) server_negotiation.cc:407] Connection header received
0809 04:44:15.676884 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:15.676891 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:15.677090 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:15.677257 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:15.678180 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.679061 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.680347 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.680835 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.680955 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.684383 (+ 3428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:15.684393 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:15.684405 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:15.684444 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:15.686872 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:15.687538 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:15.687544 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:15.687546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:15.687657 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:15.687995 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:15.687998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:15.688000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:15.688397 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:15.688685 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:15.688958 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 04:44:15.689228 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":237,"threads_started":1}
I20260809 04:44:15.690737 19495 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:15.763005 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 19365
I20260809 04:44:15.780802 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 19505
Aug 09 04:44:15 dist-test-slave-1jjb krb5kdc[19112](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.3@KRBTEST.COM
I20260809 04:44:15.795118 19662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:15.781631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46552 (local address 127.12.234.3:44037)
0809 04:44:15.782150 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:15.782157 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:15.782183 (+ 26us) server_negotiation.cc:407] Connection header received
0809 04:44:15.782238 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:15.782243 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:15.782433 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:15.782596 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:15.783668 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.784759 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.785927 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:15.786486 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:15.786629 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:15.790419 (+ 3790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:15.790429 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:15.790442 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:15.790468 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:15.792983 (+ 2515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:15.793526 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:15.793532 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:15.793535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:15.793649 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:15.793938 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:15.793940 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:15.793942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:15.794240 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:15.794449 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:15.794604 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 04:44:15.794850 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":243,"threads_started":1}
I20260809 04:44:15.796361 19635 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:15.872005 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 19505
I20260809 04:44:15.893536 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 19125
I20260809 04:44:15.894665 19192 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:15.998028 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 19125
2026-08-09T04:44:16Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3708 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:16 dist-test-slave-1jjb krb5kdc[19667](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:16 dist-test-slave-1jjb krb5kdc[19667](info): set up 2 sockets
Aug 09 04:44:16 dist-test-slave-1jjb krb5kdc[19667](info): commencing operation
krb5kdc: starting...
W20260809 04:44:17.144237 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:44:17 dist-test-slave-1jjb krb5kdc[19667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:17Z Disabled control of system clock
WARNING: no policy specified for oryx/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.62.keytab.
Entry for principal oryx/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.62.keytab.
I20260809 04:44:17.397710 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:44473
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.62.keytab
--principal=oryx/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:39489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:44473
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:17.716878 19681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:17.717134 19681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:17.717301 19681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:17.723315 19681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:17.723425 19681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:17.723501 19681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:17.723587 19681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:17.723662 19681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:17.732331 19681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:39489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:44473
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.62.keytab
--principal=oryx/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:44473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:17.733507 19681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:17.734681 19681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:17.745818 19689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:17.745893 19686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:17.745873 19687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:17.746959 19681 server_base.cc:1034] running on GCE node
I20260809 04:44:17.747715 19681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:17.748965 19681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:17.750169 19681 hybrid_clock.cc:648] HybridClock initialized: now 1786250657750133 us; error 33 us; skew 500 ppm
I20260809 04:44:17.750519 19681 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:17 dist-test-slave-1jjb krb5kdc[19667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250657, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:17.753497 19681 init.cc:377] Logged in from keytab as oryx/127.12.234.62@KRBTEST.COM (short username oryx)
I20260809 04:44:17.754911 19681 webserver.cc:460] Webserver started at http://127.12.234.62:33267/ using document root <none> and password file <none>
I20260809 04:44:17.755558 19681 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:17.755689 19681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:17.756003 19681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:17.758869 19681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "ec11c8eec1744553802f8652681fe77a"
format_stamp: "Formatted at 2026-08-09 04:44:17 on dist-test-slave-1jjb"
server_key: "b4571f800b05100d6ab9f47189c109e1"
server_key_iv: "e36751a551c08399e25882f4f06fb890"
server_key_version: "encryptionkey@0"
I20260809 04:44:17.759549 19681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "ec11c8eec1744553802f8652681fe77a"
format_stamp: "Formatted at 2026-08-09 04:44:17 on dist-test-slave-1jjb"
server_key: "b4571f800b05100d6ab9f47189c109e1"
server_key_iv: "e36751a551c08399e25882f4f06fb890"
server_key_version: "encryptionkey@0"
I20260809 04:44:17.764348 19681 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260809 04:44:17.767414 19697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:17.768672 19681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:44:17.768842 19681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "ec11c8eec1744553802f8652681fe77a"
format_stamp: "Formatted at 2026-08-09 04:44:17 on dist-test-slave-1jjb"
server_key: "b4571f800b05100d6ab9f47189c109e1"
server_key_iv: "e36751a551c08399e25882f4f06fb890"
server_key_version: "encryptionkey@0"
I20260809 04:44:17.769030 19681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:17.816234 19681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:17.820261 19681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:17.820550 19681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:17.838367 19681 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:44473
I20260809 04:44:17.838402 19751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:44473 every 8 connection(s)
I20260809 04:44:17.839646 19681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:17.841589 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 19681
I20260809 04:44:17.841809 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:44:17.842445 13224 external_mini_cluster.cc:1428] Setting key 9e7d35aa212f3a274093de5ba3eb23cb
I20260809 04:44:17.843699 19752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 04:44:17 dist-test-slave-1jjb krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM
I20260809 04:44:17.856503 19752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a: Bootstrap starting.
I20260809 04:44:17.860380 19752 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:17.861492 19752 log.cc:824] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:17.864512 19755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:17.844370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47336 (local address 127.12.234.62:44473)
0809 04:44:17.845121 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:17.845134 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:17.845184 (+ 50us) server_negotiation.cc:407] Connection header received
0809 04:44:17.846891 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:17.846920 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:17.847403 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:17.847816 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:17.849065 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:17.851020 (+ 1955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:17.852167 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:17.853082 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:17.853266 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:17.856944 (+ 3678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:17.856955 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:17.856972 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:17.857020 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:17.860491 (+ 3471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:17.861095 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:17.861104 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:17.861115 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:17.861316 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:17.861672 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:17.861677 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:17.861679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:17.862284 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:17.862502 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:17.862986 (+ 484us) server_negotiation.cc:299] Negotiation successful
0809 04:44:17.863356 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":298,"threads_started":1}
I20260809 04:44:17.864604 19752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a: No bootstrap required, opened a new log
I20260809 04:44:17.871948 19752 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } }
I20260809 04:44:17.872334 19752 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:17.872401 19752 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec11c8eec1744553802f8652681fe77a, State: Initialized, Role: FOLLOWER
I20260809 04:44:17.872838 19752 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } }
I20260809 04:44:17.873021 19752 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:17.873112 19752 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:17.873219 19752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:17.875233 19752 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } }
I20260809 04:44:17.875698 19752 leader_election.cc:302] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec11c8eec1744553802f8652681fe77a; no voters:
I20260809 04:44:17.876251 19752 leader_election.cc:288] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:17.876358 19757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:17.877192 19757 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [term 1 LEADER]: Becoming Leader. State: Replica: ec11c8eec1744553802f8652681fe77a, State: Running, Role: LEADER
I20260809 04:44:17.877767 19757 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } }
I20260809 04:44:17.878513 19752 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:17.880355 19759 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec11c8eec1744553802f8652681fe77a. Latest consensus state: current_term: 1 leader_uuid: "ec11c8eec1744553802f8652681fe77a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } }
I20260809 04:44:17.880447 19758 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec11c8eec1744553802f8652681fe77a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } }
I20260809 04:44:17.880764 19759 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:17.881714 19758 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:17.882990 19764 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:17.887971 19764 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:17.894671 19776 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 04:44:17.897119 19764 catalog_manager.cc:1347] Generated new cluster ID: 31871136da804a259b763c92b6066c36
I20260809 04:44:17.897217 19764 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:17.907968 19764 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:17.909644 19764 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:17.920327 19764 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a: Generated new TSK 0
I20260809 04:44:17.921268 19764 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.12.234.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.234.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.234.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.1.keytab.
Entry for principal oryx/127.12.234.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.1.keytab.
I20260809 04:44:17.975029 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:0
--local_ip_for_outbound_sockets=127.12.234.1
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44473
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.1.keytab
--principal=oryx/127.12.234.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:39489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:18.273962 19780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:18.274175 19780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:18.274255 19780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:18.280431 19780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:18.280512 19780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:18.280607 19780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:44:18.288794 19780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:39489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.1.keytab
--principal=oryx/127.12.234.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44473
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:18.289846 19780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:18.290937 19780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:18.302867 19787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:18.302904 19786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:18.302939 19789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:18.303537 19780 server_base.cc:1034] running on GCE node
I20260809 04:44:18.304010 19780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:18.304809 19780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:18.305991 19780 hybrid_clock.cc:648] HybridClock initialized: now 1786250658305960 us; error 30 us; skew 500 ppm
I20260809 04:44:18.306258 19780 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:18 dist-test-slave-1jjb krb5kdc[19667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250658, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:18.309185 19780 init.cc:377] Logged in from keytab as oryx/127.12.234.1@KRBTEST.COM (short username oryx)
I20260809 04:44:18.310470 19780 webserver.cc:460] Webserver started at http://127.12.234.1:43405/ using document root <none> and password file <none>
I20260809 04:44:18.311045 19780 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:18.311174 19780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:18.311431 19780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:18.313866 19780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/data/instance:
uuid: "2833ba21f588473f8daa7d439c8d7518"
format_stamp: "Formatted at 2026-08-09 04:44:18 on dist-test-slave-1jjb"
server_key: "cb2f90894db407a06c0969f0b94b002a"
server_key_iv: "2bd4b6e02c21b3d6f5bd09f83e65cbbb"
server_key_version: "encryptionkey@0"
I20260809 04:44:18.314579 19780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance:
uuid: "2833ba21f588473f8daa7d439c8d7518"
format_stamp: "Formatted at 2026-08-09 04:44:18 on dist-test-slave-1jjb"
server_key: "cb2f90894db407a06c0969f0b94b002a"
server_key_iv: "2bd4b6e02c21b3d6f5bd09f83e65cbbb"
server_key_version: "encryptionkey@0"
I20260809 04:44:18.319063 19780 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 04:44:18.322220 19796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:18.323354 19780 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 04:44:18.323513 19780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "2833ba21f588473f8daa7d439c8d7518"
format_stamp: "Formatted at 2026-08-09 04:44:18 on dist-test-slave-1jjb"
server_key: "cb2f90894db407a06c0969f0b94b002a"
server_key_iv: "2bd4b6e02c21b3d6f5bd09f83e65cbbb"
server_key_version: "encryptionkey@0"
I20260809 04:44:18.323701 19780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:18.339867 19780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:18.343572 19780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:18.343837 19780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:18.344712 19780 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:18.345882 19780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:18.345976 19780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:18.346117 19780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:18.346180 19780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:18.376623 19780 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:46629
I20260809 04:44:18.376703 19925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:46629 every 8 connection(s)
I20260809 04:44:18.377827 19780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:44:18.383597 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 19780
I20260809 04:44:18.383832 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance
I20260809 04:44:18.384527 13224 external_mini_cluster.cc:1428] Setting key e105baa3679e2d8a462343da93612a00
Aug 09 04:44:18 dist-test-slave-1jjb krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250658, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM
I20260809 04:44:18.397774 19928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:18.380125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:49913 (local address 127.12.234.62:44473)
0809 04:44:18.380548 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:18.380552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:18.381297 (+ 745us) server_negotiation.cc:407] Connection header received
0809 04:44:18.383199 (+ 1902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:18.383205 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:18.383302 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:18.383441 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:18.385032 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.385822 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.387400 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.388184 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.388312 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:18.391290 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:18.391298 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:18.391304 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:18.391342 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:18.394536 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:18.395450 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:18.395458 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:18.395479 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:18.395632 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:18.396148 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:18.396151 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:18.396153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:18.396415 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:18.396528 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:18.397353 (+ 825us) server_negotiation.cc:299] Negotiation successful
0809 04:44:18.397534 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":227,"threads_started":1}
I20260809 04:44:18.399008 19929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:18.380488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.12.234.1:49913)
0809 04:44:18.381163 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:18.381195 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:18.382888 (+ 1693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:18.383725 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:18.383738 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:18.384356 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:18.384880 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:18.384895 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.386004 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.386011 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:18.387244 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:18.387257 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.388361 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.388366 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:18.388533 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:18.389164 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:18.389190 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:18.391116 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:18.394693 (+ 3577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:18.394698 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:18.394708 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:18.395296 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:18.395745 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:18.395751 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:18.395756 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:18.395984 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:18.396551 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:18.396561 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:18.396988 (+ 427us) client_negotiation.cc:769] Sending connection context
0809 04:44:18.397287 (+ 299us) client_negotiation.cc:240] Negotiation successful
0809 04:44:18.397822 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":268,"threads_started":1}
I20260809 04:44:18.400337 19926 heartbeater.cc:347] Connected to a master server at 127.12.234.62:44473
I20260809 04:44:18.400601 19926 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:18.401261 19926 heartbeater.cc:511] Master 127.12.234.62:44473 requested a full tablet report, sending...
I20260809 04:44:18.403169 19716 ts_manager.cc:194] Registered new tserver with Master: 2833ba21f588473f8daa7d439c8d7518 (127.12.234.1:46629)
I20260809 04:44:18.404524 19716 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.234.1@KRBTEST.COM'} at 127.12.234.1:49913
WARNING: no policy specified for oryx/127.12.234.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.234.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.234.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.2.keytab.
Entry for principal oryx/127.12.234.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.2.keytab.
I20260809 04:44:18.434779 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:0
--local_ip_for_outbound_sockets=127.12.234.2
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44473
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.2.keytab
--principal=oryx/127.12.234.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:39489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:18.732367 19932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:18.732595 19932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:18.732725 19932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:18.738929 19932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:18.739012 19932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:18.739100 19932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:44:18.747767 19932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:39489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.2.keytab
--principal=oryx/127.12.234.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44473
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:18.748833 19932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:18.750042 19932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:18.761833 19940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:18.761906 19938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:18.761909 19937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:18.762517 19932 server_base.cc:1034] running on GCE node
I20260809 04:44:18.762964 19932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:18.763748 19932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:18.764930 19932 hybrid_clock.cc:648] HybridClock initialized: now 1786250658764897 us; error 27 us; skew 500 ppm
I20260809 04:44:18.765214 19932 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:18 dist-test-slave-1jjb krb5kdc[19667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250658, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:18.768108 19932 init.cc:377] Logged in from keytab as oryx/127.12.234.2@KRBTEST.COM (short username oryx)
I20260809 04:44:18.769297 19932 webserver.cc:460] Webserver started at http://127.12.234.2:37503/ using document root <none> and password file <none>
I20260809 04:44:18.769920 19932 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:18.770069 19932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:18.770364 19932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:18.772918 19932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/data/instance:
uuid: "de0ed2dbaf5d438eae07171c965a3142"
format_stamp: "Formatted at 2026-08-09 04:44:18 on dist-test-slave-1jjb"
server_key: "7ad2a491331e656e09a945a3b1d4934c"
server_key_iv: "7065e821cb3a8890da16624ea2de4924"
server_key_version: "encryptionkey@0"
I20260809 04:44:18.773700 19932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance:
uuid: "de0ed2dbaf5d438eae07171c965a3142"
format_stamp: "Formatted at 2026-08-09 04:44:18 on dist-test-slave-1jjb"
server_key: "7ad2a491331e656e09a945a3b1d4934c"
server_key_iv: "7065e821cb3a8890da16624ea2de4924"
server_key_version: "encryptionkey@0"
I20260809 04:44:18.778241 19932 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:44:18.781345 19947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:18.782641 19932 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:44:18.782805 19932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "de0ed2dbaf5d438eae07171c965a3142"
format_stamp: "Formatted at 2026-08-09 04:44:18 on dist-test-slave-1jjb"
server_key: "7ad2a491331e656e09a945a3b1d4934c"
server_key_iv: "7065e821cb3a8890da16624ea2de4924"
server_key_version: "encryptionkey@0"
I20260809 04:44:18.782994 19932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:18.795965 19932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:18.799448 19932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:18.799710 19932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:18.800554 19932 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:18.801702 19932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:18.801800 19932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:18.801934 19932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:18.802006 19932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:18.833618 19932 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:34979
I20260809 04:44:18.833659 20065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:34979 every 8 connection(s)
I20260809 04:44:18.834837 19932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:44:18.844868 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 19932
I20260809 04:44:18.845017 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance
I20260809 04:44:18.845526 13224 external_mini_cluster.cc:1428] Setting key 50f88ebb19344f4423836f899bfeb966
Aug 09 04:44:18 dist-test-slave-1jjb krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250658, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM
I20260809 04:44:18.855875 19928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:18.837263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:60299 (local address 127.12.234.62:44473)
0809 04:44:18.837469 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:18.837475 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:18.838795 (+ 1320us) server_negotiation.cc:407] Connection header received
0809 04:44:18.840576 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:18.840582 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:18.840669 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:18.840801 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:18.842176 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.843221 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.844629 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.845143 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.845275 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:18.849528 (+ 4253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:18.849544 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:18.849549 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:18.849584 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:18.852805 (+ 3221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:18.853701 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:18.853706 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:18.853710 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:18.853858 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:18.854369 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:18.854374 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:18.854376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:18.854694 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:18.854807 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:18.855526 (+ 719us) server_negotiation.cc:299] Negotiation successful
0809 04:44:18.855678 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:44:18.857265 20068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:18.837801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.12.234.2:60299)
0809 04:44:18.838638 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:18.838680 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:18.840279 (+ 1599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:18.841005 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:18.841012 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:18.841434 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:18.842012 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:18.842030 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.843389 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.843392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:18.844495 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:18.844506 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:18.845353 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:18.845362 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:18.845606 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:18.846307 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:18.846332 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:18.849265 (+ 2933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:18.852950 (+ 3685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:18.852960 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:18.852976 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:18.853551 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:18.853976 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:18.853990 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:18.853995 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:18.854250 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:18.854812 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:18.854820 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:18.855202 (+ 382us) client_negotiation.cc:769] Sending connection context
0809 04:44:18.855459 (+ 257us) client_negotiation.cc:240] Negotiation successful
0809 04:44:18.855930 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":379,"threads_started":1}
I20260809 04:44:18.858649 20066 heartbeater.cc:347] Connected to a master server at 127.12.234.62:44473
I20260809 04:44:18.858932 20066 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:18.859597 20066 heartbeater.cc:511] Master 127.12.234.62:44473 requested a full tablet report, sending...
I20260809 04:44:18.860998 19716 ts_manager.cc:194] Registered new tserver with Master: de0ed2dbaf5d438eae07171c965a3142 (127.12.234.2:34979)
I20260809 04:44:18.861845 19716 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.234.2@KRBTEST.COM'} at 127.12.234.2:60299
WARNING: no policy specified for oryx/127.12.234.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.234.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.234.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.3.keytab.
Entry for principal oryx/127.12.234.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.3.keytab.
I20260809 04:44:18.896682 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:0
--local_ip_for_outbound_sockets=127.12.234.3
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44473
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.3.keytab
--principal=oryx/127.12.234.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:39489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:19.208269 20071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:19.208560 20071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:19.208721 20071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:19.215234 20071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:19.215380 20071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:19.215552 20071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:44:19.224038 20071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:39489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.3.keytab
--principal=oryx/127.12.234.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44473
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:19.225159 20071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:19.226301 20071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:19.238394 20076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:19.238428 20079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:19.238462 20077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:19.239356 20071 server_base.cc:1034] running on GCE node
I20260809 04:44:19.239900 20071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:19.240828 20071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:19.242017 20071 hybrid_clock.cc:648] HybridClock initialized: now 1786250659241981 us; error 28 us; skew 500 ppm
I20260809 04:44:19.242297 20071 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:19 dist-test-slave-1jjb krb5kdc[19667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250659, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:19.248492 20071 init.cc:377] Logged in from keytab as oryx/127.12.234.3@KRBTEST.COM (short username oryx)
I20260809 04:44:19.249907 20071 webserver.cc:460] Webserver started at http://127.12.234.3:44715/ using document root <none> and password file <none>
I20260809 04:44:19.250496 20071 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:19.250591 20071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:19.250883 20071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:19.253562 20071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/data/instance:
uuid: "571f385b7e1143a5a32f5862960236c0"
format_stamp: "Formatted at 2026-08-09 04:44:19 on dist-test-slave-1jjb"
server_key: "1802c467be4735d8816bc2cc35d9e6e2"
server_key_iv: "3df1e4d08ec386c3343617535665c833"
server_key_version: "encryptionkey@0"
I20260809 04:44:19.254292 20071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance:
uuid: "571f385b7e1143a5a32f5862960236c0"
format_stamp: "Formatted at 2026-08-09 04:44:19 on dist-test-slave-1jjb"
server_key: "1802c467be4735d8816bc2cc35d9e6e2"
server_key_iv: "3df1e4d08ec386c3343617535665c833"
server_key_version: "encryptionkey@0"
I20260809 04:44:19.259160 20071 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:44:19.262243 20087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:19.263423 20071 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:44:19.263592 20071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "571f385b7e1143a5a32f5862960236c0"
format_stamp: "Formatted at 2026-08-09 04:44:19 on dist-test-slave-1jjb"
server_key: "1802c467be4735d8816bc2cc35d9e6e2"
server_key_iv: "3df1e4d08ec386c3343617535665c833"
server_key_version: "encryptionkey@0"
I20260809 04:44:19.263767 20071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:19.277045 20071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:19.280838 20071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:19.281085 20071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:19.281878 20071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:19.283049 20071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:19.283152 20071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:19.283293 20071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:19.283371 20071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:19.315099 20071 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:42793
I20260809 04:44:19.315169 20206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:42793 every 8 connection(s)
I20260809 04:44:19.316249 20071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:44:19.325026 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 20071
I20260809 04:44:19.325224 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance
I20260809 04:44:19.325739 13224 external_mini_cluster.cc:1428] Setting key 3228ee4d946d1ff2ab41e8e61ff3ccc8
Aug 09 04:44:19 dist-test-slave-1jjb krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250659, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM
I20260809 04:44:19.335012 19928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.318699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:45115 (local address 127.12.234.62:44473)
0809 04:44:19.318876 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:19.318880 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:19.319857 (+ 977us) server_negotiation.cc:407] Connection header received
0809 04:44:19.321602 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:19.321608 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:19.321693 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:19.321801 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:19.323147 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.323785 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.325612 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.326224 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.326354 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.330309 (+ 3955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:19.330313 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:19.330316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:19.330343 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:19.332551 (+ 2208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:19.333217 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:19.333221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:19.333223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:19.333351 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:19.333734 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:19.333737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:19.333739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:19.333967 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:19.334062 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:19.334649 (+ 587us) server_negotiation.cc:299] Negotiation successful
0809 04:44:19.334782 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:44:19.335894 20209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.319067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.12.234.3:45115)
0809 04:44:19.319716 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:19.319755 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:19.321324 (+ 1569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:19.322019 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:19.322026 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:19.322487 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:19.322986 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.323000 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.323923 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.323934 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.325479 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.325496 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.326566 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.326578 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.326995 (+ 417us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.327790 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:19.327838 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:19.330104 (+ 2266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:19.332686 (+ 2582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:19.332693 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:19.332703 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:19.333091 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:19.333455 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:19.333457 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:19.333460 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:19.333637 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:19.334062 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:19.334067 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:19.334350 (+ 283us) client_negotiation.cc:769] Sending connection context
0809 04:44:19.334561 (+ 211us) client_negotiation.cc:240] Negotiation successful
0809 04:44:19.334910 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":255,"threads_started":1}
I20260809 04:44:19.337275 20207 heartbeater.cc:347] Connected to a master server at 127.12.234.62:44473
I20260809 04:44:19.337539 20207 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:19.338249 20207 heartbeater.cc:511] Master 127.12.234.62:44473 requested a full tablet report, sending...
I20260809 04:44:19.339596 19716 ts_manager.cc:194] Registered new tserver with Master: 571f385b7e1143a5a32f5862960236c0 (127.12.234.3:42793)
I20260809 04:44:19.340613 19716 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.234.3@KRBTEST.COM'} at 127.12.234.3:45115
I20260809 04:44:19.341117 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.12.234.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.234.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.234.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.61.keytab.
Entry for principal oryx/127.12.234.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.61.keytab.
I20260809 04:44:19.391530 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.61:41005
--webserver_interface=127.12.234.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.61.keytab
--principal=oryx/127.12.234.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:39489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:44473,127.12.234.61:41005
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:44:19.407073 19926 heartbeater.cc:503] Master 127.12.234.62:44473 was elected leader, sending a full tablet report...
W20260809 04:44:19.710973 20212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:19.711247 20212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:19.711395 20212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:19.717607 20212 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:19.717743 20212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:19.717834 20212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:19.717918 20212 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:19.717988 20212 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:19.726238 20212 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:39489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:44473,127.12.234.61:41005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.61.keytab
--principal=oryx/127.12.234.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.61:41005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.234.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
W20260809 04:44:19.726562 20212 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.12.234.62:44473,127.12.234.61:41005'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 04:44:19.727555 20212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:19.728631 20212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:19.739151 20218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:19.739185 20220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:19.739217 20217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:19.740097 20212 server_base.cc:1034] running on GCE node
I20260809 04:44:19.740789 20212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:19.741999 20212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:19.743211 20212 hybrid_clock.cc:648] HybridClock initialized: now 1786250659743137 us; error 63 us; skew 500 ppm
I20260809 04:44:19.743570 20212 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:19 dist-test-slave-1jjb krb5kdc[19667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250659, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:19.746433 20212 init.cc:377] Logged in from keytab as oryx/127.12.234.61@KRBTEST.COM (short username oryx)
I20260809 04:44:19.747865 20212 webserver.cc:460] Webserver started at http://127.12.234.61:36793/ using document root <none> and password file <none>
I20260809 04:44:19.748512 20212 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:19.748631 20212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:19.748910 20212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:19.751685 20212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-1/data/instance:
uuid: "b4a48d1bc3cb43ef90c0916f666eec83"
format_stamp: "Formatted at 2026-08-09 04:44:19 on dist-test-slave-1jjb"
server_key: "55ab6ef6a68c3b7b39e35147e5af754f"
server_key_iv: "95d5259c7616b4e94885347b999d578c"
server_key_version: "encryptionkey@0"
I20260809 04:44:19.752435 20212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-1/wal/instance:
uuid: "b4a48d1bc3cb43ef90c0916f666eec83"
format_stamp: "Formatted at 2026-08-09 04:44:19 on dist-test-slave-1jjb"
server_key: "55ab6ef6a68c3b7b39e35147e5af754f"
server_key_iv: "95d5259c7616b4e94885347b999d578c"
server_key_version: "encryptionkey@0"
I20260809 04:44:19.756959 20212 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 04:44:19.760052 20227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:19.761242 20212 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:44:19.761413 20212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-1/wal
uuid: "b4a48d1bc3cb43ef90c0916f666eec83"
format_stamp: "Formatted at 2026-08-09 04:44:19 on dist-test-slave-1jjb"
server_key: "55ab6ef6a68c3b7b39e35147e5af754f"
server_key_iv: "95d5259c7616b4e94885347b999d578c"
server_key_version: "encryptionkey@0"
I20260809 04:44:19.761626 20212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:19.778422 20212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:19.781888 20212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:19.782125 20212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:19.799726 20212 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.61:41005
I20260809 04:44:19.799759 20282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.61:41005 every 8 connection(s)
I20260809 04:44:19.800974 20212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-1/data/info.pb
I20260809 04:44:19.804481 20283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:19.810307 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 20212
I20260809 04:44:19.810385 20283 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:44:19 dist-test-slave-1jjb krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250659, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.61@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM
Aug 09 04:44:19 dist-test-slave-1jjb krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.234.61@KRBTEST.COM
I20260809 04:44:19.829399 19928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.812908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47348 (local address 127.12.234.62:44473)
0809 04:44:19.813110 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:19.813117 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:19.813818 (+ 701us) server_negotiation.cc:407] Connection header received
0809 04:44:19.814972 (+ 1154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:19.814976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:19.815034 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:19.815113 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:19.816460 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.817231 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.818844 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.819270 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.819365 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.822774 (+ 3409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:19.822786 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:19.822789 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:19.822824 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:19.825881 (+ 3057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:19.826882 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:19.826888 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:19.826893 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:19.827066 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:19.827584 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:19.827591 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:19.827595 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:19.827987 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:19.828169 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:19.828795 (+ 626us) server_negotiation.cc:299] Negotiation successful
0809 04:44:19.828990 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 04:44:19.830251 20286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.812193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51574 (local address 127.12.234.61:41005)
0809 04:44:19.812900 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:19.812911 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:19.812950 (+ 39us) server_negotiation.cc:407] Connection header received
0809 04:44:19.814706 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:19.814731 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:19.815013 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:19.815197 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:19.816543 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.817920 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.819025 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.819543 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.819716 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.823511 (+ 3795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:19.823524 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:19.823539 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:19.823579 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:19.826925 (+ 3346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:19.827625 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:19.827633 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:19.827641 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:19.827811 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:19.828177 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:19.828181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:19.828184 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:19.828469 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:19.828776 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:19.829026 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 04:44:19.829311 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":106,"server-negotiator.queue_time_us":519,"thread_start_us":321,"threads_started":1}
I20260809 04:44:19.830295 20287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.812904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.0.0.1:47348)
0809 04:44:19.813643 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:19.813691 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:19.814706 (+ 1015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:19.815193 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:19.815199 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:19.815879 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:19.816275 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.816292 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.817418 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.817424 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.818713 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.818724 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.819478 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.819485 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.819685 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.820485 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:19.820503 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:19.822535 (+ 2032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:19.826026 (+ 3491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:19.826036 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:19.826044 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:19.826759 (+ 715us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:19.827199 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:19.827205 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:19.827210 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:19.827447 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:19.828148 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:19.828155 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:19.828447 (+ 292us) client_negotiation.cc:769] Sending connection context
0809 04:44:19.828730 (+ 283us) client_negotiation.cc:240] Negotiation successful
0809 04:44:19.829194 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"mutex_wait_us":45,"thread_start_us":361,"threads_started":1}
I20260809 04:44:19.832031 20283 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:44:19 dist-test-slave-1jjb krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250659, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.61@KRBTEST.COM for oryx/127.12.234.61@KRBTEST.COM
I20260809 04:44:19.845345 20287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.832959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:41005 (local address 127.0.0.1:51590)
0809 04:44:19.833141 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:19.833153 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:19.833260 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:19.833714 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:19.833718 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:19.834115 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:19.834278 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.834288 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.835149 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.835154 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.836361 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.836384 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.837204 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.837210 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.837440 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.838062 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:19.838100 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:19.840036 (+ 1936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:19.843176 (+ 3140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:19.843179 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:19.843181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:19.843515 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:19.843911 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:19.843916 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:19.843919 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:19.844029 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:19.844519 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:19.844523 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:19.844649 (+ 126us) client_negotiation.cc:769] Sending connection context
0809 04:44:19.844778 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 04:44:19.845052 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117}
I20260809 04:44:19.845548 20286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.833019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51590 (local address 127.12.234.61:41005)
0809 04:44:19.833431 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:19.833435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:19.833448 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:44:19.833484 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:19.833488 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:19.833566 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:19.833655 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:19.834411 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.834994 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.836566 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.837051 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.837152 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.840228 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:19.840234 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:19.840238 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:19.840269 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:19.843059 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:19.843647 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:19.843684 (+ 37us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:19.843689 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:19.843803 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:19.844178 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:19.844187 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:19.844190 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:19.844413 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:19.844547 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:19.844900 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 04:44:19.845157 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":98,"server-negotiator.queue_time_us":307}
I20260809 04:44:19.850037 20283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83: Bootstrap starting.
I20260809 04:44:19.853155 20283 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:19.854105 20283 log.cc:824] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:19.856583 20283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83: No bootstrap required, opened a new log
I20260809 04:44:19.862785 20283 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } }
I20260809 04:44:19.863085 20283 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:19.863185 20283 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4a48d1bc3cb43ef90c0916f666eec83, State: Initialized, Role: FOLLOWER
I20260809 04:44:19.863672 20283 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } }
I20260809 04:44:19.864140 20066 heartbeater.cc:503] Master 127.12.234.62:44473 was elected leader, sending a full tablet report...
I20260809 04:44:19.864519 20289 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } } }
I20260809 04:44:19.864848 20289 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:44:19.865612 20283 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 04:44:19.873852 20300 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:44:19.873976 20300 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:44:19.876650 20212 master_runner.cc:428] Detected that this master b4a48d1bc3cb43ef90c0916f666eec83 is joining an existing cluster
I20260809 04:44:19.876758 20212 master_runner.cc:432] Initiating AddMaster RPC to add 127.12.234.61:41005
I20260809 04:44:19.895285 19928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.880502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47360 (local address 127.12.234.62:44473)
0809 04:44:19.880805 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:19.880812 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:19.882012 (+ 1200us) server_negotiation.cc:407] Connection header received
0809 04:44:19.882162 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:19.882166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:19.882228 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:19.882329 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:19.883347 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.884299 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.885443 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.885966 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.886070 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.888046 (+ 1976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:19.888051 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:19.888055 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:19.888113 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:19.891584 (+ 3471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:19.892436 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:19.892443 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:19.892446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:19.892611 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:19.893080 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:19.893102 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:19.893106 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:19.893452 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:19.893838 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:19.894474 (+ 636us) server_negotiation.cc:299] Negotiation successful
0809 04:44:19.894944 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192}
I20260809 04:44:19.896310 20307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.881300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.0.0.1:47360)
0809 04:44:19.881923 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:19.881935 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:19.882052 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:19.882348 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:19.882352 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:19.882897 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:19.883196 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.883207 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.884456 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.884459 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.885308 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.885316 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.886122 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.886129 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.886307 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.886971 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:19.886993 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:19.887898 (+ 905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:19.891721 (+ 3823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:19.891730 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:19.891734 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:19.892286 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:19.892763 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:19.892769 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:19.892773 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:19.892947 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:19.893946 (+ 999us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:19.893952 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:19.894141 (+ 189us) client_negotiation.cc:769] Sending connection context
0809 04:44:19.895563 (+ 1422us) client_negotiation.cc:240] Negotiation successful
0809 04:44:19.895851 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":341,"threads_started":1}
I20260809 04:44:19.900697 20306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.880523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:41005 (local address 127.0.0.1:51592)
0809 04:44:19.881204 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:19.881223 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:19.881351 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:19.881840 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:19.881844 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:19.882535 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:19.882915 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.882935 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.888204 (+ 5269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.888210 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.889721 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.889737 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.890861 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.890866 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.891025 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.891531 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:19.891549 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:19.892655 (+ 1106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:19.896618 (+ 3963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:19.896622 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:19.896625 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:19.897180 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:19.897579 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:19.897583 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:19.897585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:19.897767 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:19.898433 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:19.898436 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:19.898565 (+ 129us) client_negotiation.cc:769] Sending connection context
0809 04:44:19.898692 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 04:44:19.898936 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":371,"threads_started":1}
I20260809 04:44:19.901746 20286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.880459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51592 (local address 127.12.234.61:41005)
0809 04:44:19.880666 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:19.880671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:19.881317 (+ 646us) server_negotiation.cc:407] Connection header received
0809 04:44:19.881505 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:19.881509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:19.881576 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:19.881677 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:19.883076 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.884116 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.889874 (+ 5758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.890542 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.890644 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.892823 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:19.892827 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:19.892831 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:19.892863 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:19.896450 (+ 3587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:19.897325 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:19.897329 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:19.897332 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:19.897464 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:19.897876 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:19.897880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:19.897884 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:19.898246 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:19.898388 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:19.901300 (+ 2912us) server_negotiation.cc:299] Negotiation successful
0809 04:44:19.901493 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
WARNING: no policy specified for oryx/127.12.234.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.234.60@KRBTEST.COM" created.
Aug 09 04:44:19 dist-test-slave-1jjb krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250657, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.62@KRBTEST.COM for oryx/127.12.234.61@KRBTEST.COM
I20260809 04:44:19.921250 20286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.902637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51606 (local address 127.12.234.61:41005)
0809 04:44:19.903465 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:19.903473 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:19.905048 (+ 1575us) server_negotiation.cc:407] Connection header received
0809 04:44:19.905355 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:19.905359 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:19.905482 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:19.905581 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:19.906489 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.907161 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.908393 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.908812 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.908940 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.914727 (+ 5787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:19.914733 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:19.914737 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:19.914771 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:19.918110 (+ 3339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:19.919035 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:19.919040 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:19.919043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:19.919180 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:19.919617 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:19.919621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:19.919624 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:19.919930 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:19.920059 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:19.920660 (+ 601us) server_negotiation.cc:299] Negotiation successful
0809 04:44:19.920862 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":1715,"threads_started":1}
I20260809 04:44:19.921307 20309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:19.902325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:41005 (local address 127.0.0.1:51606)
0809 04:44:19.904945 (+ 2620us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:19.904975 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:19.905163 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:19.905627 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:19.905633 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:19.906156 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:19.906325 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.906332 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.907296 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.907301 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.908272 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:19.908283 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:19.908962 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:19.908967 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:19.909179 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:19.909750 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:19.909766 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:19.914532 (+ 4766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:19.918258 (+ 3726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:19.918272 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:19.918281 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:19.918887 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:19.919300 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:19.919305 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:19.919310 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:19.919481 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:19.920088 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:19.920098 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:19.920303 (+ 205us) client_negotiation.cc:769] Sending connection context
0809 04:44:19.920556 (+ 253us) client_negotiation.cc:240] Negotiation successful
0809 04:44:19.920862 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2441,"thread_start_us":2318,"threads_started":1}
I20260809 04:44:19.923385 19716 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.12.234.61:41005: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83"
member_type: NON_VOTER
last_known_addr {
host: "127.12.234.61"
port: 41005
}
attrs {
promote: true
}
}
dest_uuid: "ec11c8eec1744553802f8652681fe77a"
cas_config_opid_index: -1
I20260809 04:44:19.924417 19716 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: NON_VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: true } }
I20260809 04:44:19.936241 20315 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER b4a48d1bc3cb43ef90c0916f666eec83 (127.12.234.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: NON_VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: true } } }
I20260809 04:44:19.937532 20255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:19.940868 20255 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [term 1 FOLLOWER]: Refusing update from remote peer ec11c8eec1744553802f8652681fe77a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 04:44:19.940647 20315 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.12.234.61:41005
I20260809 04:44:19.942955 20212 master.cc:537] Master@127.12.234.61:41005 shutting down...
I20260809 04:44:19.942667 20314 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ec11c8eec1744553802f8652681fe77a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: NON_VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: true } } }
I20260809 04:44:19.942565 20316 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: NON_VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 04:44:19.943080 20314 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [sys.catalog]: This master's current role is: LEADER
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.234.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.60.keytab.
Entry for principal oryx/127.12.234.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.60.keytab.
I20260809 04:44:19.948159 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.60:41485
--webserver_interface=127.12.234.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.60.keytab
--principal=oryx/127.12.234.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:39489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:44473,127.12.234.61:41005,127.12.234.60:41485
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:19.948729 19700 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260809 04:44:19.960410 20212 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:44:19.960917 20212 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:19.961150 20212 tablet_replica.cc:330] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83: stopping tablet replica
W20260809 04:44:19.962625 19700 consensus_peers.cc:487] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a -> Peer b4a48d1bc3cb43ef90c0916f666eec83 (127.12.234.61:41005): Couldn't send request to peer b4a48d1bc3cb43ef90c0916f666eec83. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 04:44:19.978058 20212 master.cc:552] Master@127.12.234.61:41005 shutdown complete.
I20260809 04:44:19.978286 20212 master_runner.cc:305] Clearing existing system tablet
I20260809 04:44:19.978820 20212 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:19.978956 20212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:19.984186 20212 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:44:19.986246 20321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:19.986663 20212 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:19.987063 20212 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:44:19.987206 20212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-1/wal
uuid: "b4a48d1bc3cb43ef90c0916f666eec83"
format_stamp: "Formatted at 2026-08-09 04:44:19 on dist-test-slave-1jjb"
server_key: "55ab6ef6a68c3b7b39e35147e5af754f"
server_key_iv: "95d5259c7616b4e94885347b999d578c"
server_key_version: "encryptionkey@0"
I20260809 04:44:19.993068 20212 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:19.999243 20212 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 04:44:19.999765 20212 log.cc:1196] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 04:44:20.000247 20212 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83: Deleting consensus metadata
I20260809 04:44:20.000545 20212 master_runner.cc:315] Copying system tablet from 127.12.234.62:44473
I20260809 04:44:20.002961 20212 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83: tablet copy: Beginning tablet copy session from remote peer at address 127.12.234.62:44473
I20260809 04:44:20.022439 20327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:20.003928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.0.0.1:47376)
0809 04:44:20.008505 (+ 4577us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:20.008524 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:20.008700 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:20.009145 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:20.009149 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:20.009671 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:20.009995 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:20.010008 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.011445 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.011450 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:20.013082 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:20.013100 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.014030 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.014038 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:20.014246 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:20.014988 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:20.015019 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:20.016181 (+ 1162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:20.019854 (+ 3673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:20.019858 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:20.019861 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:20.020313 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:20.020690 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:20.020694 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:20.020697 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:20.020889 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:20.021463 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:20.021468 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:20.021625 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 04:44:20.021772 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 04:44:20.022037 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4436,"thread_start_us":4299,"threads_started":1}
I20260809 04:44:20.022490 19928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:20.003914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47376 (local address 127.12.234.62:44473)
0809 04:44:20.004120 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:20.004128 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:20.008664 (+ 4536us) server_negotiation.cc:407] Connection header received
0809 04:44:20.008839 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:20.008846 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:20.008950 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:20.009082 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:20.010172 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.011284 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.013261 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.013856 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.013976 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:20.016383 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:20.016391 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:20.016397 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:20.016433 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:20.019715 (+ 3282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:20.020435 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:20.020440 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:20.020443 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:20.020572 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:20.021005 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:20.021010 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:20.021014 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:20.021332 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:20.021473 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:20.021910 (+ 437us) server_negotiation.cc:299] Negotiation successful
0809 04:44:20.022198 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 04:44:20.023689 19737 tablet_copy_service.cc:140] P ec11c8eec1744553802f8652681fe77a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b4a48d1bc3cb43ef90c0916f666eec83 ({username='oryx', principal='oryx/127.12.234.61@KRBTEST.COM'} at 127.0.0.1:47376)
I20260809 04:44:20.023986 19737 tablet_copy_service.cc:161] P ec11c8eec1744553802f8652681fe77a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b4a48d1bc3cb43ef90c0916f666eec83 at {username='oryx', principal='oryx/127.12.234.61@KRBTEST.COM'} at 127.0.0.1:47376: session id = b4a48d1bc3cb43ef90c0916f666eec83-00000000000000000000000000000000
I20260809 04:44:20.028174 19737 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 04:44:20.030308 20212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 04:44:20.036732 20212 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83: tablet copy: Starting download of 0 data blocks...
I20260809 04:44:20.037079 20212 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83: tablet copy: Starting download of 1 WAL segments...
I20260809 04:44:20.039774 20212 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 04:44:20.043998 19737 tablet_copy_service.cc:342] P ec11c8eec1744553802f8652681fe77a: Request end of tablet copy session b4a48d1bc3cb43ef90c0916f666eec83-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.12.234.61@KRBTEST.COM'} at 127.0.0.1:47376
I20260809 04:44:20.044265 19737 tablet_copy_service.cc:433] P ec11c8eec1744553802f8652681fe77a: ending tablet copy session b4a48d1bc3cb43ef90c0916f666eec83-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b4a48d1bc3cb43ef90c0916f666eec83
I20260809 04:44:20.055752 20212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:20.058326 20328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:20.059592 20329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:20.063692 20212 server_base.cc:1034] running on GCE node
W20260809 04:44:20.064123 20331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:20.064563 20212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:20.065171 20212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:20.066524 20212 hybrid_clock.cc:648] HybridClock initialized: now 1786250660066461 us; error 31 us; skew 500 ppm
I20260809 04:44:20.066737 20212 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:20 dist-test-slave-1jjb krb5kdc[19667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250660, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:20.069291 20212 init.cc:377] Logged in from keytab as oryx/127.12.234.61@KRBTEST.COM (short username oryx)
I20260809 04:44:20.072485 20212 webserver.cc:460] Webserver started at http://127.12.234.61:44813/ using document root <none> and password file <none>
I20260809 04:44:20.072854 20212 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:20.072942 20212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:20.076100 20212 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:44:20.078200 20339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:20.078660 20212 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260809 04:44:20.078890 20212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-1/wal
uuid: "b4a48d1bc3cb43ef90c0916f666eec83"
format_stamp: "Formatted at 2026-08-09 04:44:19 on dist-test-slave-1jjb"
server_key: "55ab6ef6a68c3b7b39e35147e5af754f"
server_key_iv: "95d5259c7616b4e94885347b999d578c"
server_key_version: "encryptionkey@0"
I20260809 04:44:20.079152 20212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:20.096869 20212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:20.101372 20212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:20.124951 20212 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.61:41005
I20260809 04:44:20.124995 20392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.61:41005 every 8 connection(s)
I20260809 04:44:20.126278 20212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-1/data/info.pb
I20260809 04:44:20.131184 20393 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:44:20.134699 20393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83: Bootstrap starting.
I20260809 04:44:20.150038 20393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:44:20.150507 20393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83: Bootstrap complete.
I20260809 04:44:20.151489 20393 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: NON_VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: true } }
I20260809 04:44:20.151729 20393 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b4a48d1bc3cb43ef90c0916f666eec83, State: Initialized, Role: LEARNER
I20260809 04:44:20.152029 20393 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: NON_VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: true } }
I20260809 04:44:20.153160 20393 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:20.153002 20396 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: NON_VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: true } } }
I20260809 04:44:20.153486 20396 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [sys.catalog]: This master's current role is: LEARNER
I20260809 04:44:20.161031 20408 catalog_manager.cc:1259] Loaded cluster ID: 31871136da804a259b763c92b6066c36
I20260809 04:44:20.161134 20408 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83: loading cluster ID for follower catalog manager: success
I20260809 04:44:20.163376 20408 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83: acquiring CA information for follower catalog manager: success
I20260809 04:44:20.165225 20408 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 04:44:20.331044 20319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:20.331347 20319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:20.331488 20319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:20.337965 20319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:20.338083 20319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:20.338168 20319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:20.338249 20319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:20.338318 20319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:20.343412 20207 heartbeater.cc:503] Master 127.12.234.62:44473 was elected leader, sending a full tablet report...
I20260809 04:44:20.347028 20319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:39489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:44473,127.12.234.61:41005,127.12.234.60:41485
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.60.keytab
--principal=oryx/127.12.234.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.60:41485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.234.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:20.348279 20319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:20.349450 20319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:20.360770 20413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:20.360707 20414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:20.360745 20416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:20.361673 20319 server_base.cc:1034] running on GCE node
I20260809 04:44:20.362360 20319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:20.363591 20319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:20.364794 20319 hybrid_clock.cc:648] HybridClock initialized: now 1786250660364751 us; error 33 us; skew 500 ppm
I20260809 04:44:20.365108 20319 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:20 dist-test-slave-1jjb krb5kdc[19667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250660, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:20.368214 20319 init.cc:377] Logged in from keytab as oryx/127.12.234.60@KRBTEST.COM (short username oryx)
I20260809 04:44:20.369603 20319 webserver.cc:460] Webserver started at http://127.12.234.60:43517/ using document root <none> and password file <none>
I20260809 04:44:20.370260 20319 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:20.370412 20319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:20.370685 20319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:20.373822 20319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-2/data/instance:
uuid: "f12602d2831a44c397103d20328e6b17"
format_stamp: "Formatted at 2026-08-09 04:44:20 on dist-test-slave-1jjb"
server_key: "cd0d8139f3b08f91171a70a25a1c6dd2"
server_key_iv: "5bdb604c141d8145af1712f010f66717"
server_key_version: "encryptionkey@0"
I20260809 04:44:20.374552 20319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-2/wal/instance:
uuid: "f12602d2831a44c397103d20328e6b17"
format_stamp: "Formatted at 2026-08-09 04:44:20 on dist-test-slave-1jjb"
server_key: "cd0d8139f3b08f91171a70a25a1c6dd2"
server_key_iv: "5bdb604c141d8145af1712f010f66717"
server_key_version: "encryptionkey@0"
I20260809 04:44:20.380251 20319 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 04:44:20.383507 20423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:20.384773 20319 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:44:20.384969 20319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-2/wal
uuid: "f12602d2831a44c397103d20328e6b17"
format_stamp: "Formatted at 2026-08-09 04:44:20 on dist-test-slave-1jjb"
server_key: "cd0d8139f3b08f91171a70a25a1c6dd2"
server_key_iv: "5bdb604c141d8145af1712f010f66717"
server_key_version: "encryptionkey@0"
I20260809 04:44:20.385170 20319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:20.404266 20319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:20.407737 20319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:20.407985 20319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:20.425463 20319 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.60:41485
I20260809 04:44:20.425498 20479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.60:41485 every 8 connection(s)
I20260809 04:44:20.426640 20319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-2/data/info.pb
I20260809 04:44:20.430063 20480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:20.435622 20480 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:44:20.436930 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 20319
Aug 09 04:44:20 dist-test-slave-1jjb krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250660, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.60@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM
Aug 09 04:44:20 dist-test-slave-1jjb krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.234.60@KRBTEST.COM
I20260809 04:44:20.473308 19928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:20.438166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47392 (local address 127.12.234.62:44473)
0809 04:44:20.438334 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:20.438340 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:20.451340 (+ 13000us) server_negotiation.cc:407] Connection header received
0809 04:44:20.452969 (+ 1629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:20.452975 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:20.453048 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:20.453144 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:20.455811 (+ 2667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.456918 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.458906 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.459696 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.459762 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:20.464425 (+ 4663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:20.464433 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:20.464437 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:20.464476 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:20.468902 (+ 4426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:20.469935 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:20.469941 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:20.469944 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:20.470185 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:20.470796 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:20.470801 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:20.470803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:20.471287 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:20.471395 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:20.472526 (+ 1131us) server_negotiation.cc:299] Negotiation successful
0809 04:44:20.472811 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 04:44:20.474174 20484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:20.438609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.0.0.1:47392)
0809 04:44:20.451204 (+ 12595us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:20.451243 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:20.452674 (+ 1431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:20.453214 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:20.453222 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:20.454642 (+ 1420us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:20.455629 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:20.455656 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.457136 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.457141 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:20.458731 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:20.458742 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.460257 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.460265 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:20.460488 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:20.461265 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:20.461294 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:20.464062 (+ 2768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:20.469025 (+ 4963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:20.469038 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:20.469063 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:20.469811 (+ 748us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:20.470421 (+ 610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:20.470430 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:20.470433 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:20.470664 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:20.471421 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:20.471428 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:20.471904 (+ 476us) client_negotiation.cc:769] Sending connection context
0809 04:44:20.472245 (+ 341us) client_negotiation.cc:240] Negotiation successful
0809 04:44:20.472669 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12363,"mutex_wait_us":35,"thread_start_us":12175,"threads_started":1}
I20260809 04:44:20.474793 20485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:20.438667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56820 (local address 127.12.234.60:41485)
0809 04:44:20.450329 (+ 11662us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:20.450343 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:20.451166 (+ 823us) server_negotiation.cc:407] Connection header received
0809 04:44:20.452652 (+ 1486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:20.452671 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:20.452950 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:20.453186 (+ 236us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:20.454854 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.456697 (+ 1843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.458565 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.459806 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.460012 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:20.465935 (+ 5923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:20.465960 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:20.465973 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:20.466006 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:20.471156 (+ 5150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:20.471859 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:20.471865 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:20.471867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:20.472050 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:20.472521 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:20.472526 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:20.472528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:20.473071 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:20.473738 (+ 667us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:20.473893 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 04:44:20.474154 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":11415,"spinlock_wait_cycles":15744,"thread_start_us":11228,"threads_started":1}
I20260809 04:44:20.475697 20480 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:44:20 dist-test-slave-1jjb krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250660, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.60@KRBTEST.COM for oryx/127.12.234.61@KRBTEST.COM
I20260809 04:44:20.491784 20486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:20.476783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51618 (local address 127.12.234.61:41005)
0809 04:44:20.477888 (+ 1105us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:20.477894 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:20.477914 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:44:20.477974 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:20.477980 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:20.478062 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:20.478218 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:20.479013 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.480558 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.481993 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.482441 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.482525 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:20.486072 (+ 3547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:20.486076 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:20.486078 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:20.486106 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:20.489464 (+ 3358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:20.490146 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:20.490149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:20.490151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:20.490343 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:20.490704 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:20.490707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:20.490708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:20.490970 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:20.491101 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:20.491368 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 04:44:20.491509 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":965,"thread_start_us":797,"threads_started":1}
I20260809 04:44:20.491932 20484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:20.477085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:41005 (local address 127.0.0.1:51618)
0809 04:44:20.477463 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:20.477480 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:20.477569 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:20.478279 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:20.478285 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:20.478747 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:20.478899 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:20.478907 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.480686 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.480689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:20.481882 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:20.481893 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.482549 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.482552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:20.482802 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:20.483328 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:20.483342 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:20.485896 (+ 2554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:20.489585 (+ 3689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:20.489588 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:20.489591 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:20.490049 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:20.490429 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:20.490432 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:20.490434 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:20.490619 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:20.491071 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:20.491073 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:20.491198 (+ 125us) client_negotiation.cc:769] Sending connection context
0809 04:44:20.491289 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 04:44:20.491518 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262}
I20260809 04:44:20.493206 20480 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.60" port: 41485 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:44:20 dist-test-slave-1jjb krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250660, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.60@KRBTEST.COM for oryx/127.12.234.60@KRBTEST.COM
I20260809 04:44:20.507766 20485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:20.494216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56830 (local address 127.12.234.60:41485)
0809 04:44:20.494336 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:20.494339 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:20.494350 (+ 11us) server_negotiation.cc:407] Connection header received
0809 04:44:20.494380 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:20.494382 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:20.494421 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:20.494530 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:20.495194 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.496197 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.497476 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.498114 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.498244 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:20.501748 (+ 3504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:20.501753 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:20.501756 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:20.501780 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:20.504786 (+ 3006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:20.505458 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:20.505461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:20.505463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:20.505673 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:20.506055 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:20.506059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:20.506062 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:20.506341 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:20.506475 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:20.506948 (+ 473us) server_negotiation.cc:299] Negotiation successful
0809 04:44:20.507274 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":133,"server-negotiator.queue_time_us":50}
I20260809 04:44:20.507766 20484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:20.493817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:41485 (local address 127.0.0.1:56830)
0809 04:44:20.493953 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:20.493966 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:20.494032 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:20.494545 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:20.494549 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:20.494911 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:20.495072 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:20.495080 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.496306 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.496313 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:20.497333 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:20.497344 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.498251 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.498255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:20.498606 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:20.499134 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:20.499147 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:20.501559 (+ 2412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:20.504906 (+ 3347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:20.504912 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:20.504915 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:20.505324 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:20.505786 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:20.505791 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:20.505794 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:20.505955 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:20.506445 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:20.506449 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:20.506601 (+ 152us) client_negotiation.cc:769] Sending connection context
0809 04:44:20.506796 (+ 195us) client_negotiation.cc:240] Negotiation successful
0809 04:44:20.507103 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":2}
I20260809 04:44:20.515309 20480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17: Bootstrap starting.
I20260809 04:44:20.517082 20486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:20.509069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51630 (local address 127.12.234.61:41005)
0809 04:44:20.509259 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:20.509265 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:20.509989 (+ 724us) server_negotiation.cc:407] Connection header received
0809 04:44:20.510222 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:20.510226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:20.510348 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:20.510425 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:20.511090 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.511751 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.514651 (+ 2900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.515864 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.515987 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:20.516232 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:20.516654 (+ 422us) server_negotiation.cc:299] Negotiation successful
0809 04:44:20.516774 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:44:20.517395 20488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:20.508957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:41005 (local address 127.0.0.1:51630)
0809 04:44:20.509855 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:20.509875 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:20.510090 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:20.510495 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:20.510517 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:20.510544 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:20.510953 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:20.510966 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.511917 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.511932 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:20.514448 (+ 2516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:20.514462 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.516140 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.516147 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:20.516464 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:20.516512 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 04:44:20.516633 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 04:44:20.516778 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"thread_start_us":568,"threads_started":1}
I20260809 04:44:20.519212 20367 raft_consensus.cc:1203] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260809 04:44:20.519915 20396 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec11c8eec1744553802f8652681fe77a. Latest consensus state: current_term: 1 leader_uuid: "ec11c8eec1744553802f8652681fe77a" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: NON_VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: true } } }
I20260809 04:44:20.520280 20396 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [sys.catalog]: This master's current role is: LEARNER
I20260809 04:44:20.521093 20480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:20.522760 20480 log.cc:824] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:20.525751 20480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17: No bootstrap required, opened a new log
I20260809 04:44:20.536176 20480 raft_consensus.cc:348] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 41485 } }
I20260809 04:44:20.536674 20480 raft_consensus.cc:374] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:20.536909 20480 raft_consensus.cc:729] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f12602d2831a44c397103d20328e6b17, State: Initialized, Role: FOLLOWER
I20260809 04:44:20.537539 20480 consensus_queue.cc:260] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 41485 } }
I20260809 04:44:20.538849 20490 sys_catalog.cc:455] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 41485 } } }
I20260809 04:44:20.539460 20490 sys_catalog.cc:458] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:44:20.539606 20480 sys_catalog.cc:564] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 04:44:20.557269 20502 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:44:20.557677 20502 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:44:20.561462 20319 master_runner.cc:428] Detected that this master f12602d2831a44c397103d20328e6b17 is joining an existing cluster
I20260809 04:44:20.561673 20319 master_runner.cc:432] Initiating AddMaster RPC to add 127.12.234.60:41485
I20260809 04:44:20.581732 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 19780
I20260809 04:44:20.604802 19928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:20.571032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47408 (local address 127.12.234.62:44473)
0809 04:44:20.571285 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:20.571292 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:20.574379 (+ 3087us) server_negotiation.cc:407] Connection header received
0809 04:44:20.574697 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:20.574705 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:20.574793 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:20.574881 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:20.578119 (+ 3238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.579675 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.588544 (+ 8869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.589854 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.589986 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:20.594272 (+ 4286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:20.594280 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:20.594285 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:20.594325 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:20.600220 (+ 5895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:20.601082 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:20.601089 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:20.601092 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:20.601320 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:20.601899 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:20.601904 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:20.601912 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:20.602393 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:20.602542 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:20.603518 (+ 976us) server_negotiation.cc:299] Negotiation successful
0809 04:44:20.603822 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
Aug 09 04:44:20 dist-test-slave-1jjb krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.234.1@KRBTEST.COM
I20260809 04:44:20.606312 20509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:20.572008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.0.0.1:47408)
0809 04:44:20.574258 (+ 2250us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:20.574278 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:20.574535 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:20.575171 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:20.575178 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:20.576544 (+ 1366us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:20.577924 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:20.577945 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.580061 (+ 2116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.580064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:20.588413 (+ 8349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:20.588428 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.590440 (+ 2012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.590453 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:20.590811 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:20.592055 (+ 1244us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:20.592110 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:20.594014 (+ 1904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:20.600367 (+ 6353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:20.600371 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:20.600375 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:20.600938 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:20.601443 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:20.601448 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:20.601451 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:20.601743 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:20.602613 (+ 870us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:20.602624 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:20.603110 (+ 486us) client_negotiation.cc:769] Sending connection context
0809 04:44:20.603324 (+ 214us) client_negotiation.cc:240] Negotiation successful
0809 04:44:20.604139 (+ 815us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1657,"thread_start_us":7153,"threads_started":1}
I20260809 04:44:20.616808 20486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:20.571357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51646 (local address 127.12.234.61:41005)
0809 04:44:20.571598 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:20.571605 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:20.581488 (+ 9883us) server_negotiation.cc:407] Connection header received
0809 04:44:20.581969 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:20.581975 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:20.582059 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:20.582154 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:20.591484 (+ 9330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.592842 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.596273 (+ 3431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.597605 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.597731 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:20.605475 (+ 7744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:20.605482 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:20.605487 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:20.605525 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:20.611677 (+ 6152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:20.612722 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:20.612727 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:20.612730 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:20.613020 (+ 290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:20.613629 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:20.613636 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:20.613638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:20.614128 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:20.614264 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:20.615604 (+ 1340us) server_negotiation.cc:299] Negotiation successful
0809 04:44:20.616014 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 04:44:20.618673 20512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:20.583267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54438 (local address 127.12.234.1:46629)
0809 04:44:20.589791 (+ 6524us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:20.589800 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:20.589840 (+ 40us) server_negotiation.cc:407] Connection header received
0809 04:44:20.589945 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:20.589954 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:20.590222 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:20.592562 (+ 2340us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:20.593031 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.595163 (+ 2132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.597886 (+ 2723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.599142 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.599942 (+ 800us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:20.607458 (+ 7516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:20.607473 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:20.607490 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:20.607526 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:20.613247 (+ 5721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:20.614363 (+ 1116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:20.614371 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:20.614374 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:20.614611 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:20.615121 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:20.615126 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:20.615128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:20.615607 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:20.617436 (+ 1829us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:20.617654 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 04:44:20.618069 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6311,"thread_start_us":6979,"threads_started":1}
I20260809 04:44:20.618893 20508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:20.572533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:41005 (local address 127.0.0.1:51646)
0809 04:44:20.581280 (+ 8747us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:20.581329 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:20.581512 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:20.582418 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:20.582424 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:20.589580 (+ 7156us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:20.591321 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:20.591342 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.593068 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.593074 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:20.596044 (+ 2970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:20.596066 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.598283 (+ 2217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.598292 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:20.598738 (+ 446us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:20.599697 (+ 959us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:20.599725 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:20.605262 (+ 5537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:20.611877 (+ 6615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:20.611884 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:20.611896 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:20.612554 (+ 658us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:20.613164 (+ 610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:20.613169 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:20.613172 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:20.613455 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:20.614338 (+ 883us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:20.614343 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:20.614727 (+ 384us) client_negotiation.cc:769] Sending connection context
0809 04:44:20.614907 (+ 180us) client_negotiation.cc:240] Negotiation successful
0809 04:44:20.617489 (+ 2582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8119,"thread_start_us":748,"threads_started":1}
I20260809 04:44:20.620712 19919 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:20.642745 20511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:20.573807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:41485 (local address 127.0.0.1:56846)
0809 04:44:20.600963 (+ 27156us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:20.600981 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:20.601166 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:20.601707 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:20.601711 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:20.604758 (+ 3047us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:20.608270 (+ 3512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:20.608295 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.615701 (+ 7406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.615706 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:20.626013 (+ 10307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:20.626029 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.628629 (+ 2600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.628637 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:20.628900 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:20.630513 (+ 1613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:20.630536 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:20.632891 (+ 2355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:20.639250 (+ 6359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:20.639255 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:20.639259 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:20.639874 (+ 615us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:20.640296 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:20.640300 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:20.640304 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:20.640501 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:20.641366 (+ 865us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:20.641370 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:20.641588 (+ 218us) client_negotiation.cc:769] Sending connection context
0809 04:44:20.641710 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 04:44:20.642029 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26213,"thread_start_us":25415,"threads_started":1}
I20260809 04:44:20.644213 20485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:20.573882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56846 (local address 127.12.234.60:41485)
0809 04:44:20.574074 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:20.574081 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:20.601053 (+ 26972us) server_negotiation.cc:407] Connection header received
0809 04:44:20.601323 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:20.601328 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:20.601398 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:20.601460 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:20.608459 (+ 6999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.611119 (+ 2660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.626884 (+ 15765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.628015 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.628126 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:20.633170 (+ 5044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:20.633177 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:20.633185 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:20.633225 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:20.639123 (+ 5898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:20.639984 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:20.639988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:20.639991 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:20.640194 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:20.640607 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:20.640611 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:20.640615 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:20.641178 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:20.641304 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:20.643115 (+ 1811us) server_negotiation.cc:299] Negotiation successful
0809 04:44:20.643353 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 04:44:20 dist-test-slave-1jjb krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250657, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.62@KRBTEST.COM for oryx/127.12.234.60@KRBTEST.COM
I20260809 04:44:20.656783 20488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:20.615056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:41485 (local address 127.0.0.1:56852)
0809 04:44:20.615194 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:20.615212 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:20.615331 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:20.634133 (+ 18802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:20.634139 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:20.634894 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:20.635276 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:20.635293 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.639416 (+ 4123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.639421 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:20.641466 (+ 2045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:20.641481 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.642853 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.642860 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:20.643118 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:20.643871 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:20.643907 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:20.647817 (+ 3910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:20.653279 (+ 5462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:20.653284 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:20.653287 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:20.654042 (+ 755us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:20.654507 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:20.654511 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:20.654515 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:20.654755 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:20.655461 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:20.655466 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:20.655731 (+ 265us) client_negotiation.cc:769] Sending connection context
0809 04:44:20.655902 (+ 171us) client_negotiation.cc:240] Negotiation successful
0809 04:44:20.656209 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 04:44:20.656867 20513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:20.615395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56852 (local address 127.12.234.60:41485)
0809 04:44:20.633641 (+ 18246us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:20.633647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:20.633699 (+ 52us) server_negotiation.cc:407] Connection header received
0809 04:44:20.633795 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:20.633800 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:20.633896 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:20.636130 (+ 2234us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:20.637000 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.639226 (+ 2226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.641714 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.642647 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.643551 (+ 904us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:20.648174 (+ 4623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:20.648182 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:20.648187 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:20.648218 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:20.653158 (+ 4940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:20.654211 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:20.654216 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:20.654219 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:20.654392 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:20.654862 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:20.654866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:20.654869 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:20.655297 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:20.655422 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:20.656025 (+ 603us) server_negotiation.cc:299] Negotiation successful
0809 04:44:20.656232 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17969,"thread_start_us":19203,"threads_started":1}
I20260809 04:44:20.658694 19716 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.12.234.60:41485: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f12602d2831a44c397103d20328e6b17"
member_type: NON_VOTER
last_known_addr {
host: "127.12.234.60"
port: 41485
}
attrs {
promote: true
}
}
dest_uuid: "ec11c8eec1744553802f8652681fe77a"
cas_config_opid_index: 5
I20260809 04:44:20.659790 19716 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: NON_VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: true } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: NON_VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: true } }
W20260809 04:44:20.660156 20429 tablet.cc:2369] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:44:20.661813 20487 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f12602d2831a44c397103d20328e6b17 (127.12.234.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: NON_VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: true } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: NON_VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: true } } }
I20260809 04:44:20.662607 20367 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [term 1 LEARNER]: Refusing update from remote peer ec11c8eec1744553802f8652681fe77a: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 04:44:20.663385 20453 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:20.668804 20453 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [term 1 FOLLOWER]: Refusing update from remote peer ec11c8eec1744553802f8652681fe77a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 04:44:20.671963 20515 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: NON_VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260809 04:44:20.676231 20487 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.12.234.60:41485
I20260809 04:44:20.677088 20515 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ec11c8eec1744553802f8652681fe77a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: NON_VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: true } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: NON_VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: true } } }
I20260809 04:44:20.675814 20516 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: NON_VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.002s
I20260809 04:44:20.678103 20515 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:20.688990 20319 master.cc:537] Master@127.12.234.60:41485 shutting down...
I20260809 04:44:20.692886 20367 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f12602d2831a44c397103d20328e6b17 (127.12.234.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: NON_VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: true } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: NON_VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: true } } }
I20260809 04:44:20.703706 20396 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec11c8eec1744553802f8652681fe77a" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: NON_VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: true } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: NON_VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: true } } }
I20260809 04:44:20.704229 20396 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [sys.catalog]: This master's current role is: LEARNER
I20260809 04:44:20.705459 20517 raft_consensus.cc:1050] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a: attempting to promote NON_VOTER b4a48d1bc3cb43ef90c0916f666eec83 to VOTER
I20260809 04:44:20.706503 20517 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: NON_VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: true } }
I20260809 04:44:20.710776 20367 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [term 1 LEARNER]: Refusing update from remote peer ec11c8eec1744553802f8652681fe77a: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 04:44:20.711589 20517 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260809 04:44:20.712205 19700 consensus_peers.cc:487] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a -> Peer f12602d2831a44c397103d20328e6b17 (127.12.234.60:41485): Couldn't send request to peer f12602d2831a44c397103d20328e6b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 04:44:20.714300 20516 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b4a48d1bc3cb43ef90c0916f666eec83 (127.12.234.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: NON_VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: true } } }
I20260809 04:44:20.715221 20367 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b4a48d1bc3cb43ef90c0916f666eec83 (127.12.234.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: NON_VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: true } } }
I20260809 04:44:20.720275 20515 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ec11c8eec1744553802f8652681fe77a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: NON_VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: true } } }
I20260809 04:44:20.720635 20515 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:20.720667 20396 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec11c8eec1744553802f8652681fe77a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: NON_VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: true } } }
I20260809 04:44:20.720901 20396 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:44:20.721096 20515 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec11c8eec1744553802f8652681fe77a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: NON_VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: true } } }
I20260809 04:44:20.721418 20515 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:20.734728 20319 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:44:20.735142 20319 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:20.735343 20319 tablet_replica.cc:330] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17: stopping tablet replica
I20260809 04:44:20.743432 20319 master.cc:552] Master@127.12.234.60:41485 shutdown complete.
I20260809 04:44:20.743544 20319 master_runner.cc:305] Clearing existing system tablet
I20260809 04:44:20.743947 20319 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:20.744050 20319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:20.747852 20319 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:44:20.750092 20522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:20.750507 20319 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:20.750770 20319 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 04:44:20.750867 20319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-2/wal
uuid: "f12602d2831a44c397103d20328e6b17"
format_stamp: "Formatted at 2026-08-09 04:44:20 on dist-test-slave-1jjb"
server_key: "cd0d8139f3b08f91171a70a25a1c6dd2"
server_key_iv: "5bdb604c141d8145af1712f010f66717"
server_key_version: "encryptionkey@0"
I20260809 04:44:20.756124 20319 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:20.761958 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 19780
I20260809 04:44:20.765506 20319 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 04:44:20.765816 20319 log.cc:1196] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 04:44:20.766831 20319 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17: Deleting consensus metadata
I20260809 04:44:20.767169 20319 master_runner.cc:315] Copying system tablet from 127.12.234.62:44473
I20260809 04:44:20.774694 20319 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17: tablet copy: Beginning tablet copy session from remote peer at address 127.12.234.62:44473
I20260809 04:44:20.784581 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:46629
--local_ip_for_outbound_sockets=127.12.234.1
--tserver_master_addrs=127.12.234.62:44473
--webserver_port=43405
--webserver_interface=127.12.234.1
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.1.keytab
--principal=oryx/127.12.234.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:39489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.12.234.62:44473,127.12.234.61:41005
--tserver_master_addrs=127.12.234.62:44473,127.12.234.61:41005,127.12.234.60:41485 with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:44:20.792184 20528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:20.775485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.0.0.1:47420)
0809 04:44:20.777031 (+ 1546us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:20.777052 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:20.777311 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:20.777781 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:20.777785 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:20.778326 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:20.778580 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:20.778592 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.780254 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.780260 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:20.781651 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:20.781665 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.782904 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.782912 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:20.783090 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:20.784268 (+ 1178us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:20.784298 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:20.785017 (+ 719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:20.789406 (+ 4389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:20.789411 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:20.789415 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:20.789776 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:20.790226 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:20.790230 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:20.790245 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:20.790388 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:20.790923 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:20.790928 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:20.791099 (+ 171us) client_negotiation.cc:769] Sending connection context
0809 04:44:20.791236 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 04:44:20.791574 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1291,"thread_start_us":1132,"threads_started":1}
I20260809 04:44:20.792266 19928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:20.775563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47420 (local address 127.12.234.62:44473)
0809 04:44:20.776300 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:20.776308 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:20.777151 (+ 843us) server_negotiation.cc:407] Connection header received
0809 04:44:20.777491 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:20.777501 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:20.777598 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:20.777724 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:20.778727 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.780064 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.781807 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:20.782702 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:20.782873 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:20.785230 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:20.785235 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:20.785238 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:20.785265 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:20.789258 (+ 3993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:20.789903 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:20.789909 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:20.789913 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:20.790105 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:20.790490 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:20.790495 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:20.790497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:20.790802 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:20.791012 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:20.791339 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 04:44:20.791629 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569}
I20260809 04:44:20.793176 19737 tablet_copy_service.cc:140] P ec11c8eec1744553802f8652681fe77a: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f12602d2831a44c397103d20328e6b17 ({username='oryx', principal='oryx/127.12.234.60@KRBTEST.COM'} at 127.0.0.1:47420)
I20260809 04:44:20.793334 19737 tablet_copy_service.cc:161] P ec11c8eec1744553802f8652681fe77a: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f12602d2831a44c397103d20328e6b17 at {username='oryx', principal='oryx/127.12.234.60@KRBTEST.COM'} at 127.0.0.1:47420: session id = f12602d2831a44c397103d20328e6b17-00000000000000000000000000000000
I20260809 04:44:20.796737 19737 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 04:44:20.798599 20319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 04:44:20.805017 20319 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17: tablet copy: Starting download of 0 data blocks...
I20260809 04:44:20.805384 20319 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17: tablet copy: Starting download of 1 WAL segments...
I20260809 04:44:20.808140 20319 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 04:44:20.812252 19737 tablet_copy_service.cc:342] P ec11c8eec1744553802f8652681fe77a: Request end of tablet copy session f12602d2831a44c397103d20328e6b17-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.12.234.60@KRBTEST.COM'} at 127.0.0.1:47420
I20260809 04:44:20.812386 19737 tablet_copy_service.cc:433] P ec11c8eec1744553802f8652681fe77a: ending tablet copy session f12602d2831a44c397103d20328e6b17-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f12602d2831a44c397103d20328e6b17
I20260809 04:44:20.822762 20319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:20.825369 20530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:20.838894 20531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:20.841748 20319 server_base.cc:1034] running on GCE node
W20260809 04:44:20.842435 20533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:20.842934 20319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:20.855420 20319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:20.856686 20319 hybrid_clock.cc:648] HybridClock initialized: now 1786250660856644 us; error 34 us; skew 500 ppm
I20260809 04:44:20.857043 20319 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:20 dist-test-slave-1jjb krb5kdc[19667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250660, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:20.859407 20319 init.cc:377] Logged in from keytab as oryx/127.12.234.60@KRBTEST.COM (short username oryx)
I20260809 04:44:20.863253 20319 webserver.cc:460] Webserver started at http://127.12.234.60:41743/ using document root <none> and password file <none>
I20260809 04:44:20.863691 20319 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:20.863853 20319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:20.868462 20319 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 04:44:20.870584 20540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:20.870998 20319 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260809 04:44:20.871153 20319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-2/wal
uuid: "f12602d2831a44c397103d20328e6b17"
format_stamp: "Formatted at 2026-08-09 04:44:20 on dist-test-slave-1jjb"
server_key: "cd0d8139f3b08f91171a70a25a1c6dd2"
server_key_iv: "5bdb604c141d8145af1712f010f66717"
server_key_version: "encryptionkey@0"
I20260809 04:44:20.871421 20319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:20.901309 20319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:20.907663 20319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:21.012945 20319 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.60:41485
I20260809 04:44:21.012928 20599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.60:41485 every 8 connection(s)
I20260809 04:44:21.014066 20319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/master-2/data/info.pb
I20260809 04:44:21.019495 20600 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:44:21.023501 20600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17: Bootstrap starting.
I20260809 04:44:21.047477 20600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:44:21.047994 20600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17: Bootstrap complete.
I20260809 04:44:21.049352 20600 raft_consensus.cc:348] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: NON_VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: true } }
I20260809 04:44:21.049715 20600 raft_consensus.cc:729] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f12602d2831a44c397103d20328e6b17, State: Initialized, Role: LEARNER
I20260809 04:44:21.050056 20600 consensus_queue.cc:260] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: NON_VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: true } }
I20260809 04:44:21.051160 20604 sys_catalog.cc:455] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: NON_VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: true } } }
I20260809 04:44:21.051622 20600 sys_catalog.cc:564] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:21.051642 20604 sys_catalog.cc:458] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [sys.catalog]: This master's current role is: LEARNER
I20260809 04:44:21.065308 20615 catalog_manager.cc:1259] Loaded cluster ID: 31871136da804a259b763c92b6066c36
I20260809 04:44:21.065424 20615 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17: loading cluster ID for follower catalog manager: success
I20260809 04:44:21.069281 20615 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17: acquiring CA information for follower catalog manager: success
I20260809 04:44:21.071584 20615 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 04:44:21.195757 20529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:21.196031 20529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:21.196199 20529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:21.202672 20529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:21.202801 20529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:21.202963 20529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:44:21.212044 20529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:39489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.1.keytab
--principal=oryx/127.12.234.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:46629
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=43405
--tserver_master_addrs=127.12.234.62:44473,127.12.234.61:41005,127.12.234.60:41485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:21.213413 20529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:21.214639 20529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:21.227346 20620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:21.227368 20623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:21.227425 20621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:21.229499 20529 server_base.cc:1034] running on GCE node
I20260809 04:44:21.230391 20529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:21.231688 20529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:21.232960 20529 hybrid_clock.cc:648] HybridClock initialized: now 1786250661232863 us; error 64 us; skew 500 ppm
I20260809 04:44:21.233405 20529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:21 dist-test-slave-1jjb krb5kdc[19667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250661, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:21.238699 20529 init.cc:377] Logged in from keytab as oryx/127.12.234.1@KRBTEST.COM (short username oryx)
I20260809 04:44:21.240597 20529 webserver.cc:460] Webserver started at http://127.12.234.1:43405/ using document root <none> and password file <none>
I20260809 04:44:21.241379 20529 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:21.241612 20529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:21.247637 20529 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 04:44:21.251252 20631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:21.252863 20529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:44:21.253113 20529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "2833ba21f588473f8daa7d439c8d7518"
format_stamp: "Formatted at 2026-08-09 04:44:18 on dist-test-slave-1jjb"
server_key: "cb2f90894db407a06c0969f0b94b002a"
server_key_iv: "2bd4b6e02c21b3d6f5bd09f83e65cbbb"
server_key_version: "encryptionkey@0"
I20260809 04:44:21.254554 20529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:21.269172 20633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:21.259114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:41485 (local address 127.0.0.1:56864)
0809 04:44:21.260056 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:21.260100 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:21.260305 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:21.263432 (+ 3127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:21.263436 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:21.263461 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:21.263657 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:21.263674 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:21.265375 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:21.265380 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:21.266658 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:21.266671 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:21.268272 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:21.268277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:21.268447 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:21.268508 (+ 61us) client_negotiation.cc:769] Sending connection context
0809 04:44:21.268604 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 04:44:21.268755 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"thread_start_us":579,"threads_started":1}
I20260809 04:44:21.269461 20634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:21.259244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56864 (local address 127.12.234.60:41485)
0809 04:44:21.263139 (+ 3895us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:21.263143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:21.263159 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:44:21.263216 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:21.263220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:21.263283 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:21.263417 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:21.263801 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:21.265181 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:21.266833 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:21.268042 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:21.268246 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:21.268565 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:21.268621 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 04:44:21.268792 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3759,"thread_start_us":3550,"threads_started":1}
I20260809 04:44:21.270431 20570 raft_consensus.cc:1203] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 04:44:21.271169 20604 sys_catalog.cc:455] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec11c8eec1744553802f8652681fe77a. Latest consensus state: current_term: 1 leader_uuid: "ec11c8eec1744553802f8652681fe77a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: NON_VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: true } } }
I20260809 04:44:21.271519 20604 sys_catalog.cc:458] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [sys.catalog]: This master's current role is: LEARNER
I20260809 04:44:21.274164 20529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:21.280215 20529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:21.280717 20529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:21.282390 20529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:21.284687 20529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:21.284864 20529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:21.285015 20529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:21.285151 20529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:21.372610 20529 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:46629
I20260809 04:44:21.373013 20754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:46629 every 8 connection(s)
I20260809 04:44:21.374277 20529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:44:21.382076 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 20529
I20260809 04:44:21.382493 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 19932
Aug 09 04:44:21 dist-test-slave-1jjb krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250661, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM
Aug 09 04:44:21 dist-test-slave-1jjb krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250661, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.60@KRBTEST.COM
I20260809 04:44:21.418274 20759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:21.378213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:50875 (local address 127.12.234.62:44473)
0809 04:44:21.380168 (+ 1955us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:21.380176 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:21.382704 (+ 2528us) server_negotiation.cc:407] Connection header received
0809 04:44:21.388155 (+ 5451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:21.388164 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:21.388314 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:21.388421 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:21.392263 (+ 3842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:21.394198 (+ 1935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:21.397801 (+ 3603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:21.398737 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:21.398840 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:21.407090 (+ 8250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:21.407096 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:21.407100 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:21.407157 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:21.412613 (+ 5456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:21.414138 (+ 1525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:21.414155 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:21.414159 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:21.414433 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:21.415100 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:21.415105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:21.415108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:21.415430 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:21.415553 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:21.417311 (+ 1758us) server_negotiation.cc:299] Negotiation successful
0809 04:44:21.417606 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1798,"thread_start_us":1564,"threads_started":1}
Aug 09 04:44:21 dist-test-slave-1jjb krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250661, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.61@KRBTEST.COM
I20260809 04:44:21.423621 20761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:21.378845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.12.234.1:50875)
0809 04:44:21.382538 (+ 3693us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:21.382581 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:21.387777 (+ 5196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:21.389489 (+ 1712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:21.389504 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:21.390745 (+ 1241us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:21.392005 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:21.392050 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:21.394513 (+ 2463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:21.394521 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:21.397582 (+ 3061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:21.397602 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:21.399369 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:21.399379 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:21.399795 (+ 416us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:21.401593 (+ 1798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:21.401640 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:21.406769 (+ 5129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:21.412814 (+ 6045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:21.412840 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:21.412865 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:21.413932 (+ 1067us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:21.414599 (+ 667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:21.414604 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:21.414607 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:21.414972 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:21.415651 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:21.415658 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:21.416250 (+ 592us) client_negotiation.cc:769] Sending connection context
0809 04:44:21.419198 (+ 2948us) client_negotiation.cc:240] Negotiation successful
0809 04:44:21.419838 (+ 640us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3244,"thread_start_us":21165,"threads_started":1}
I20260809 04:44:21.426522 20756 heartbeater.cc:347] Connected to a master server at 127.12.234.62:44473
I20260809 04:44:21.426939 20634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:21.378187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:51889 (local address 127.12.234.60:41485)
0809 04:44:21.378430 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:21.378436 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:21.400816 (+ 22380us) server_negotiation.cc:407] Connection header received
0809 04:44:21.401018 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:21.401025 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:21.401177 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:21.401338 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:21.405635 (+ 4297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:21.407366 (+ 1731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:21.409713 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:21.410870 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:21.410970 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:21.417419 (+ 6449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:21.417426 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:21.417430 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:21.417484 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:21.422699 (+ 5215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:21.424029 (+ 1330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:21.424034 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:21.424037 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:21.424280 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:21.424770 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:21.424777 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:21.424780 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:21.425287 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:21.425444 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:21.426060 (+ 616us) server_negotiation.cc:299] Negotiation successful
0809 04:44:21.426328 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260809 04:44:21.427001 20756 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:21.428378 20760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:21.378870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:41485 (local address 127.12.234.1:51889)
0809 04:44:21.400659 (+ 21789us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:21.400698 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:21.400861 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:21.402947 (+ 2086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:21.402952 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:21.403999 (+ 1047us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:21.405463 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:21.405478 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:21.407597 (+ 2119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:21.407602 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:21.409540 (+ 1938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:21.409557 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:21.411474 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:21.411482 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:21.411727 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:21.412616 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:21.412637 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:21.417080 (+ 4443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:21.422792 (+ 5712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:21.422858 (+ 66us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:21.422862 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:21.423882 (+ 1020us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:21.424439 (+ 557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:21.424442 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:21.424446 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:21.424626 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:21.425527 (+ 901us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:21.425531 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:21.425742 (+ 211us) client_negotiation.cc:769] Sending connection context
0809 04:44:21.427174 (+ 1432us) client_negotiation.cc:240] Negotiation successful
0809 04:44:21.427512 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21593,"mutex_wait_us":37,"spinlock_wait_cycles":31872,"thread_start_us":2869,"threads_started":1}
I20260809 04:44:21.428704 20756 heartbeater.cc:511] Master 127.12.234.62:44473 requested a full tablet report, sending...
I20260809 04:44:21.429459 20755 heartbeater.cc:347] Connected to a master server at 127.12.234.60:41485
I20260809 04:44:21.429652 20755 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:21.430539 20755 heartbeater.cc:511] Master 127.12.234.60:41485 requested a full tablet report, sending...
I20260809 04:44:21.430971 19716 ts_manager.cc:194] Re-registered known tserver with Master: 2833ba21f588473f8daa7d439c8d7518 (127.12.234.1:46629)
I20260809 04:44:21.432354 19716 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.234.1@KRBTEST.COM'} at 127.12.234.1:50875
I20260809 04:44:21.432518 20560 ts_manager.cc:194] Registered new tserver with Master: 2833ba21f588473f8daa7d439c8d7518 (127.12.234.1:46629)
I20260809 04:44:21.434916 20762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:21.382988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:41005 (local address 127.12.234.1:60117)
0809 04:44:21.402079 (+ 19091us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:21.402129 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:21.402323 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:21.402758 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:21.402773 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:21.405609 (+ 2836us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:21.406633 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:21.406648 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:21.408820 (+ 2172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:21.408824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:21.413860 (+ 5036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:21.413891 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:21.415801 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:21.415807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:21.416137 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:21.417047 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:21.417068 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:21.421129 (+ 4061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:21.430995 (+ 9866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:21.431000 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:21.431004 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:21.431711 (+ 707us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:21.432450 (+ 739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:21.432463 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:21.432467 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:21.432751 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:21.433790 (+ 1039us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:21.433795 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:21.433962 (+ 167us) client_negotiation.cc:769] Sending connection context
0809 04:44:21.434099 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 04:44:21.434377 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18769,"mutex_wait_us":17,"spinlock_wait_cycles":11520,"thread_start_us":19243,"threads_started":1}
Aug 09 04:44:21 dist-test-slave-1jjb krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.234.2@KRBTEST.COM
I20260809 04:44:21.436342 20765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:21.384116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:60117 (local address 127.12.234.61:41005)
0809 04:44:21.402282 (+ 18166us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:21.402290 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:21.402362 (+ 72us) server_negotiation.cc:407] Connection header received
0809 04:44:21.402443 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:21.402451 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:21.402586 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:21.402731 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:21.406843 (+ 4112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:21.408614 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:21.414088 (+ 5474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:21.415174 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:21.415287 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:21.421472 (+ 6185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:21.421481 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:21.421486 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:21.421524 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:21.430753 (+ 9229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:21.431897 (+ 1144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:21.431903 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:21.431906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:21.432219 (+ 313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:21.433057 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:21.433062 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:21.433066 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:21.433607 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:21.433820 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:21.435237 (+ 1417us) server_negotiation.cc:299] Negotiation successful
0809 04:44:21.435518 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17970,"thread_start_us":17757,"threads_started":1}
I20260809 04:44:21.438090 20757 heartbeater.cc:347] Connected to a master server at 127.12.234.61:41005
I20260809 04:44:21.438295 20757 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:21.438972 20757 heartbeater.cc:511] Master 127.12.234.61:41005 requested a full tablet report, sending...
I20260809 04:44:21.440965 20357 ts_manager.cc:194] Registered new tserver with Master: 2833ba21f588473f8daa7d439c8d7518 (127.12.234.1:46629)
I20260809 04:44:21.444517 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:21.384104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39198 (local address 127.12.234.2:34979)
0809 04:44:21.396373 (+ 12269us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:21.396384 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:21.404356 (+ 7972us) server_negotiation.cc:407] Connection header received
0809 04:44:21.404452 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:21.404460 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:21.404775 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:21.405481 (+ 706us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:21.415208 (+ 9727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:21.418002 (+ 2794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:21.421487 (+ 3485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:21.422769 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:21.422908 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:21.437955 (+ 15047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:21.437971 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:21.437998 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:21.438067 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:21.441140 (+ 3073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:21.442209 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:21.442221 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:21.442223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:21.442359 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:21.442819 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:21.442825 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:21.442827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:21.443131 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:21.443424 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:21.443834 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 04:44:21.444150 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12022,"thread_start_us":12235,"threads_started":1}
I20260809 04:44:21.445829 20061 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:21.527098 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 19932
I20260809 04:44:21.546747 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:34979
--local_ip_for_outbound_sockets=127.12.234.2
--tserver_master_addrs=127.12.234.62:44473
--webserver_port=37503
--webserver_interface=127.12.234.2
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.2.keytab
--principal=oryx/127.12.234.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:39489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.12.234.62:44473,127.12.234.61:41005
--tserver_master_addrs=127.12.234.62:44473,127.12.234.61:41005,127.12.234.60:41485 with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:44:21.676090 20632 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec11c8eec1744553802f8652681fe77a" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: NON_VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: true } } }
I20260809 04:44:21.676564 20632 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:21.679060 20632 raft_consensus.cc:1050] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a: attempting to promote NON_VOTER f12602d2831a44c397103d20328e6b17 to VOTER
I20260809 04:44:21.679766 20632 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: false } }
I20260809 04:44:21.681818 20570 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [term 1 LEARNER]: Refusing update from remote peer ec11c8eec1744553802f8652681fe77a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 04:44:21.682672 20367 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [term 1 FOLLOWER]: Refusing update from remote peer ec11c8eec1744553802f8652681fe77a: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 04:44:21.682566 20769 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260809 04:44:21.683296 20768 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 04:44:21.686318 20768 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, f12602d2831a44c397103d20328e6b17 (127.12.234.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: false } } }
I20260809 04:44:21.687259 20367 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, f12602d2831a44c397103d20328e6b17 (127.12.234.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: false } } }
I20260809 04:44:21.691752 20768 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ec11c8eec1744553802f8652681fe77a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: false } } }
I20260809 04:44:21.692281 20632 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec11c8eec1744553802f8652681fe77a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: false } } }
I20260809 04:44:21.692587 20632 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:21.692796 20768 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:21.697453 20772 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec11c8eec1744553802f8652681fe77a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: false } } }
I20260809 04:44:21.697896 20772 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:44:21.697839 20768 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ec11c8eec1744553802f8652681fe77a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: false } } }
I20260809 04:44:21.698073 20768 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:21.698781 20570 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, f12602d2831a44c397103d20328e6b17 (127.12.234.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: false } } }
I20260809 04:44:21.701754 20604 sys_catalog.cc:455] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ec11c8eec1744553802f8652681fe77a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: false } } }
I20260809 04:44:21.702023 20604 sys_catalog.cc:458] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 04:44:21.998481 20767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:21.998711 20767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:21.998790 20767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:22.004719 20767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:22.004817 20767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:22.004913 20767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:44:22.018376 20767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:39489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.2.keytab
--principal=oryx/127.12.234.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:34979
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=37503
--tserver_master_addrs=127.12.234.62:44473,127.12.234.61:41005,127.12.234.60:41485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:22.019650 20767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:22.021028 20767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:22.033095 20778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:22.033157 20782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:22.033185 20779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:22.033953 20767 server_base.cc:1034] running on GCE node
I20260809 04:44:22.034515 20767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:22.035334 20767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:22.036545 20767 hybrid_clock.cc:648] HybridClock initialized: now 1786250662036517 us; error 40 us; skew 500 ppm
I20260809 04:44:22.036860 20767 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:22 dist-test-slave-1jjb krb5kdc[19667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250662, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:22.039769 20767 init.cc:377] Logged in from keytab as oryx/127.12.234.2@KRBTEST.COM (short username oryx)
I20260809 04:44:22.041028 20767 webserver.cc:460] Webserver started at http://127.12.234.2:37503/ using document root <none> and password file <none>
I20260809 04:44:22.041646 20767 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:22.041790 20767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:22.046458 20767 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 04:44:22.049074 20789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:22.050230 20767 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260809 04:44:22.050414 20767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "de0ed2dbaf5d438eae07171c965a3142"
format_stamp: "Formatted at 2026-08-09 04:44:18 on dist-test-slave-1jjb"
server_key: "7ad2a491331e656e09a945a3b1d4934c"
server_key_iv: "7065e821cb3a8890da16624ea2de4924"
server_key_version: "encryptionkey@0"
I20260809 04:44:22.051251 20767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:22.075971 20767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:22.079739 20767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:22.080096 20767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:22.080945 20767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:22.082168 20767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:22.082243 20767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:22.082304 20767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:22.082352 20767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:22.113838 20767 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:34979
I20260809 04:44:22.113953 20908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:34979 every 8 connection(s)
I20260809 04:44:22.115058 20767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:44:22.122664 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 20767
I20260809 04:44:22.122961 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 20071
Aug 09 04:44:22 dist-test-slave-1jjb krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250662, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM
Aug 09 04:44:22 dist-test-slave-1jjb krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250662, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.61@KRBTEST.COM
Aug 09 04:44:22 dist-test-slave-1jjb krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250662, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.60@KRBTEST.COM
I20260809 04:44:22.157070 20914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:22.117731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:43289 (local address 127.12.234.62:44473)
0809 04:44:22.121277 (+ 3546us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:22.121284 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:22.131983 (+ 10699us) server_negotiation.cc:407] Connection header received
0809 04:44:22.132959 (+ 976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:22.132968 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:22.133089 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:22.133181 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:22.137657 (+ 4476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.138864 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.141888 (+ 3024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.142608 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.142721 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:22.148234 (+ 5513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:22.148240 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:22.148244 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:22.148281 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:22.151786 (+ 3505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:22.152767 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:22.152774 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:22.152778 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:22.152925 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:22.153461 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:22.153466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:22.153471 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:22.153780 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:22.153900 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:22.156469 (+ 2569us) server_negotiation.cc:299] Negotiation successful
0809 04:44:22.156686 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3389,"thread_start_us":3260,"threads_started":1}
I20260809 04:44:22.158119 20913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:22.117721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:52097 (local address 127.12.234.61:41005)
0809 04:44:22.141883 (+ 24162us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:22.141887 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:22.141902 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:44:22.141947 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:22.141950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:22.142000 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:22.142122 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:22.142829 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.143489 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.145432 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.145988 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.146085 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:22.151036 (+ 4951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:22.151043 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:22.151047 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:22.151105 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:22.154906 (+ 3801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:22.155793 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:22.155797 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:22.155800 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:22.155929 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:22.156375 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:22.156379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:22.156382 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:22.156732 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:22.156891 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:22.157615 (+ 724us) server_negotiation.cc:299] Negotiation successful
0809 04:44:22.157813 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24072,"thread_start_us":23991,"threads_started":1}
I20260809 04:44:22.159103 20915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:22.118314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:41005 (local address 127.12.234.2:52097)
0809 04:44:22.128565 (+ 10251us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:22.128605 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:22.131129 (+ 2524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:22.142144 (+ 11015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:22.142148 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:22.142493 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:22.142692 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:22.142699 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.143636 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.143643 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:22.145268 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:22.145285 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.146362 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.146371 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:22.146573 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:22.147223 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:22.147253 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:22.150847 (+ 3594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:22.155077 (+ 4230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:22.155082 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:22.155085 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:22.155665 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:22.156048 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:22.156052 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:22.156055 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:22.156251 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:22.156883 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:22.156889 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:22.157078 (+ 189us) client_negotiation.cc:769] Sending connection context
0809 04:44:22.157371 (+ 293us) client_negotiation.cc:240] Negotiation successful
0809 04:44:22.158655 (+ 1284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9989,"mutex_wait_us":872,"thread_start_us":9775,"threads_started":1}
I20260809 04:44:22.159142 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:22.118860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.12.234.2:43289)
0809 04:44:22.131802 (+ 12942us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:22.131820 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:22.132607 (+ 787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:22.133660 (+ 1053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:22.133672 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:22.134565 (+ 893us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:22.135348 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:22.135372 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.139088 (+ 3716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.139096 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:22.141277 (+ 2181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:22.141295 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.143079 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.143087 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:22.143390 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:22.144257 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:22.144293 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:22.147959 (+ 3666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:22.151985 (+ 4026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:22.151994 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:22.152007 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:22.152590 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:22.153099 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:22.153104 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:22.153107 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:22.153328 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:22.153998 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:22.154005 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:22.156032 (+ 2027us) client_negotiation.cc:769] Sending connection context
0809 04:44:22.157282 (+ 1250us) client_negotiation.cc:240] Negotiation successful
0809 04:44:22.157804 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12816,"mutex_wait_us":4,"thread_start_us":14184,"threads_started":1}
Aug 09 04:44:22 dist-test-slave-1jjb krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.234.3@KRBTEST.COM
I20260809 04:44:22.160665 20910 heartbeater.cc:347] Connected to a master server at 127.12.234.62:44473
I20260809 04:44:22.160954 20910 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:22.161711 20910 heartbeater.cc:511] Master 127.12.234.62:44473 requested a full tablet report, sending...
I20260809 04:44:22.162557 20911 heartbeater.cc:347] Connected to a master server at 127.12.234.61:41005
I20260809 04:44:22.162674 20911 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:22.163081 20911 heartbeater.cc:511] Master 127.12.234.61:41005 requested a full tablet report, sending...
I20260809 04:44:22.163386 19716 ts_manager.cc:194] Re-registered known tserver with Master: de0ed2dbaf5d438eae07171c965a3142 (127.12.234.2:34979)
I20260809 04:44:22.164480 19716 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.234.2@KRBTEST.COM'} at 127.12.234.2:43289
I20260809 04:44:22.165305 20357 ts_manager.cc:194] Registered new tserver with Master: de0ed2dbaf5d438eae07171c965a3142 (127.12.234.2:34979)
I20260809 04:44:22.165390 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:22.129210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:33315 (local address 127.12.234.60:41485)
0809 04:44:22.144454 (+ 15244us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:22.144459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:22.144739 (+ 280us) server_negotiation.cc:407] Connection header received
0809 04:44:22.144813 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:22.144817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:22.144892 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:22.144980 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:22.147746 (+ 2766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.148790 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.150937 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.151526 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.151651 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:22.155941 (+ 4290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:22.155950 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:22.155955 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:22.155998 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:22.161939 (+ 5941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:22.163267 (+ 1328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:22.163272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:22.163275 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:22.163415 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:22.163811 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:22.163816 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:22.163819 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:22.164119 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:22.164268 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:22.164826 (+ 558us) server_negotiation.cc:299] Negotiation successful
0809 04:44:22.165044 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15129,"thread_start_us":14995,"threads_started":1}
I20260809 04:44:22.167205 20918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:22.123945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41104 (local address 127.12.234.3:42793)
0809 04:44:22.145557 (+ 21612us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:22.145564 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:22.145590 (+ 26us) server_negotiation.cc:407] Connection header received
0809 04:44:22.145648 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:22.145655 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:22.145897 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:22.146055 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:22.148439 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.150598 (+ 2159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.152846 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.153916 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.154046 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:22.160726 (+ 6680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:22.160741 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:22.160755 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:22.160802 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:22.164243 (+ 3441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:22.164986 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:22.164993 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:22.164995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:22.165138 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:22.165561 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:22.165566 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:22.165568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:22.165942 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:22.166230 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:22.166520 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 04:44:22.166790 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21427,"thread_start_us":21256,"threads_started":1}
I20260809 04:44:22.167519 20921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:22.134838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:41485 (local address 127.12.234.2:33315)
0809 04:44:22.144501 (+ 9663us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:22.144515 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:22.144642 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:22.146314 (+ 1672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:22.146321 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:22.146998 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:22.147462 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:22.147478 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.149012 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.149016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:22.150771 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:22.150787 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.151886 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.151893 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:22.152144 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:22.152775 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:22.152795 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:22.155645 (+ 2850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:22.162070 (+ 6425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:22.162076 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:22.162078 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:22.163059 (+ 981us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:22.163557 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:22.163564 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:22.163568 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:22.163702 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:22.164327 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:22.164331 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:22.164474 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 04:44:22.164719 (+ 245us) client_negotiation.cc:240] Negotiation successful
0809 04:44:22.167182 (+ 2463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9122,"thread_start_us":8975,"threads_started":1}
I20260809 04:44:22.168385 20909 heartbeater.cc:347] Connected to a master server at 127.12.234.60:41485
I20260809 04:44:22.168519 20909 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:22.168568 20202 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:22.168901 20909 heartbeater.cc:511] Master 127.12.234.60:41485 requested a full tablet report, sending...
I20260809 04:44:22.170001 20560 ts_manager.cc:194] Registered new tserver with Master: de0ed2dbaf5d438eae07171c965a3142 (127.12.234.2:34979)
I20260809 04:44:22.246850 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 20071
I20260809 04:44:22.269284 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:42793
--local_ip_for_outbound_sockets=127.12.234.3
--tserver_master_addrs=127.12.234.62:44473
--webserver_port=44715
--webserver_interface=127.12.234.3
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.3.keytab
--principal=oryx/127.12.234.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:39489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.12.234.62:44473,127.12.234.61:41005
--tserver_master_addrs=127.12.234.62:44473,127.12.234.61:41005,127.12.234.60:41485 with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:44:22.435345 20756 heartbeater.cc:503] Master 127.12.234.62:44473 was elected leader, sending a full tablet report...
W20260809 04:44:22.587673 20923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:22.587942 20923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:22.588028 20923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:22.595870 20923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:22.595954 20923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:22.596050 20923 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:44:22.604866 20923 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:39489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.3.keytab
--principal=oryx/127.12.234.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:42793
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=44715
--tserver_master_addrs=127.12.234.62:44473,127.12.234.61:41005,127.12.234.60:41485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:22.605906 20923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:22.607081 20923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:22.619490 20929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:22.619690 20928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:22.619922 20931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:22.620275 20923 server_base.cc:1034] running on GCE node
I20260809 04:44:22.620805 20923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:22.621687 20923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:22.622953 20923 hybrid_clock.cc:648] HybridClock initialized: now 1786250662622913 us; error 47 us; skew 500 ppm
I20260809 04:44:22.623262 20923 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:22 dist-test-slave-1jjb krb5kdc[19667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250662, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:22.626256 20923 init.cc:377] Logged in from keytab as oryx/127.12.234.3@KRBTEST.COM (short username oryx)
I20260809 04:44:22.627580 20923 webserver.cc:460] Webserver started at http://127.12.234.3:44715/ using document root <none> and password file <none>
I20260809 04:44:22.628341 20923 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:22.628486 20923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:22.633386 20923 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260809 04:44:22.635825 20938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:22.637084 20923 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:44:22.637257 20923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "571f385b7e1143a5a32f5862960236c0"
format_stamp: "Formatted at 2026-08-09 04:44:19 on dist-test-slave-1jjb"
server_key: "1802c467be4735d8816bc2cc35d9e6e2"
server_key_iv: "3df1e4d08ec386c3343617535665c833"
server_key_version: "encryptionkey@0"
I20260809 04:44:22.638204 20923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:22.664049 20923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:22.668378 20923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:22.668619 20923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:22.669510 20923 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:22.670779 20923 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:22.670908 20923 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:22.671020 20923 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:22.671103 20923 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:22.703069 20923 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:42793
I20260809 04:44:22.703150 21056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:42793 every 8 connection(s)
I20260809 04:44:22.704407 20923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:44:22.713070 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 20923
Aug 09 04:44:22 dist-test-slave-1jjb krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250662, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM
Aug 09 04:44:22 dist-test-slave-1jjb krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250662, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.60@KRBTEST.COM
Aug 09 04:44:22 dist-test-slave-1jjb krb5kdc[19667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250662, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.61@KRBTEST.COM
I20260809 04:44:22.733067 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:22.707182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:40151 (local address 127.12.234.62:44473)
0809 04:44:22.707614 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:22.707619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:22.708739 (+ 1120us) server_negotiation.cc:407] Connection header received
0809 04:44:22.710742 (+ 2003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:22.710751 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:22.710855 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:22.710973 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:22.713296 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.713970 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.716307 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.716752 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.716895 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:22.721268 (+ 4373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:22.721274 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:22.721278 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:22.721312 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:22.724529 (+ 3217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:22.725796 (+ 1267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:22.725801 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:22.725804 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:22.726009 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:22.730909 (+ 4900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:22.730914 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:22.730918 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:22.731264 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:22.731435 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:22.732492 (+ 1057us) server_negotiation.cc:299] Negotiation successful
0809 04:44:22.732700 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":222,"threads_started":1}
I20260809 04:44:22.736153 21063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:22.708408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:40351 (local address 127.12.234.61:41005)
0809 04:44:22.708829 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:22.708835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:22.712155 (+ 3320us) server_negotiation.cc:407] Connection header received
0809 04:44:22.712335 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:22.712342 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:22.712446 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:22.712549 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:22.714498 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.715781 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.718761 (+ 2980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.720776 (+ 2015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.720933 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:22.725089 (+ 4156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:22.725096 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:22.725100 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:22.725147 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:22.730149 (+ 5002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:22.733457 (+ 3308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:22.733464 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:22.733470 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:22.733662 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:22.734171 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:22.734178 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:22.734183 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:22.734554 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:22.734699 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:22.735313 (+ 614us) server_negotiation.cc:299] Negotiation successful
0809 04:44:22.735583 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":209,"threads_started":1}
I20260809 04:44:22.740324 21066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:22.711374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:35077 (local address 127.12.234.60:41485)
0809 04:44:22.712596 (+ 1222us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:22.712602 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:22.712636 (+ 34us) server_negotiation.cc:407] Connection header received
0809 04:44:22.712691 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:22.712696 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:22.712761 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:22.712878 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:22.714942 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.715842 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.718959 (+ 3117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.719575 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.719724 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:22.726025 (+ 6301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:22.726032 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:22.726037 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:22.726100 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:22.730534 (+ 4434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:22.737438 (+ 6904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:22.737447 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:22.737453 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:22.737640 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:22.738146 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:22.738152 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:22.738157 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:22.738564 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:22.738750 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:22.739471 (+ 721us) server_negotiation.cc:299] Negotiation successful
0809 04:44:22.739742 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1117,"thread_start_us":1011,"threads_started":1}
I20260809 04:44:22.742677 21064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:22.708398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:41005 (local address 127.12.234.3:40351)
0809 04:44:22.711094 (+ 2696us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:22.711110 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:22.712220 (+ 1110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:22.713353 (+ 1133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:22.713358 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:22.713915 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:22.714281 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:22.714295 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.716048 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.716057 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:22.718451 (+ 2394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:22.718488 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.721000 (+ 2512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.721006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:22.721763 (+ 757us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:22.722556 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:22.722576 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:22.724917 (+ 2341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:22.730917 (+ 6000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:22.730921 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:22.730924 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:22.731420 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:22.733837 (+ 2417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:22.733844 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:22.733864 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:22.734039 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:22.734787 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:22.734793 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:22.734999 (+ 206us) client_negotiation.cc:769] Sending connection context
0809 04:44:22.736669 (+ 1670us) client_negotiation.cc:240] Negotiation successful
0809 04:44:22.741692 (+ 5023us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2568,"thread_start_us":2421,"threads_started":1}
I20260809 04:44:22.742699 21065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:22.711215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:41485 (local address 127.12.234.3:35077)
0809 04:44:22.711731 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:22.711743 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:22.711820 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:22.712902 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:22.712909 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:22.714419 (+ 1510us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:22.714764 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:22.714777 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.716011 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.716017 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:22.718800 (+ 2783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:22.718815 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.719757 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.719764 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:22.719944 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:22.721297 (+ 1353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:22.721314 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:22.724783 (+ 3469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:22.736483 (+ 11700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:22.736490 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:22.736496 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:22.737256 (+ 760us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:22.737796 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:22.737804 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:22.737823 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:22.738011 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:22.738822 (+ 811us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:22.738828 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:22.739009 (+ 181us) client_negotiation.cc:769] Sending connection context
0809 04:44:22.739158 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 04:44:22.741806 (+ 2648us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":2379,"thread_start_us":283,"threads_started":1}
I20260809 04:44:22.744351 21059 heartbeater.cc:347] Connected to a master server at 127.12.234.61:41005
I20260809 04:44:22.744493 21062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:22.707702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.12.234.3:40151)
0809 04:44:22.708585 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:22.708633 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:22.710473 (+ 1840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:22.711228 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:22.711238 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:22.712180 (+ 942us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:22.713103 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:22.713139 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.714129 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.714134 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:22.716163 (+ 2029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:22.716179 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.716989 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.716997 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:22.717300 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:22.718083 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:22.718108 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:22.721071 (+ 2963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:22.725071 (+ 4000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:22.725077 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:22.725090 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:22.725686 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:22.730440 (+ 4754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:22.730449 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:22.730455 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:22.730778 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:22.731548 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:22.731559 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:22.732046 (+ 487us) client_negotiation.cc:769] Sending connection context
0809 04:44:22.732398 (+ 352us) client_negotiation.cc:240] Negotiation successful
0809 04:44:22.732863 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"mutex_wait_us":32,"thread_start_us":804,"threads_started":1}
I20260809 04:44:22.744674 21059 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:22.745774 21059 heartbeater.cc:511] Master 127.12.234.61:41005 requested a full tablet report, sending...
I20260809 04:44:22.746544 21058 heartbeater.cc:347] Connected to a master server at 127.12.234.62:44473
I20260809 04:44:22.746665 21058 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:22.746910 21057 heartbeater.cc:347] Connected to a master server at 127.12.234.60:41485
I20260809 04:44:22.747030 21057 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:22.747175 21058 heartbeater.cc:511] Master 127.12.234.62:44473 requested a full tablet report, sending...
I20260809 04:44:22.747479 21057 heartbeater.cc:511] Master 127.12.234.60:41485 requested a full tablet report, sending...
I20260809 04:44:22.747629 20357 ts_manager.cc:194] Registered new tserver with Master: 571f385b7e1143a5a32f5862960236c0 (127.12.234.3:42793)
I20260809 04:44:22.748409 19716 ts_manager.cc:194] Re-registered known tserver with Master: 571f385b7e1143a5a32f5862960236c0 (127.12.234.3:42793)
I20260809 04:44:22.748546 20560 ts_manager.cc:194] Registered new tserver with Master: 571f385b7e1143a5a32f5862960236c0 (127.12.234.3:42793)
I20260809 04:44:22.749439 19716 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.234.3@KRBTEST.COM'} at 127.12.234.3:40151
I20260809 04:44:22.758095 21074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:22.725548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57096 (local address 127.12.234.62:44473)
0809 04:44:22.731079 (+ 5531us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:22.731089 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:22.731829 (+ 740us) server_negotiation.cc:407] Connection header received
0809 04:44:22.731888 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:22.731893 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:22.731972 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:22.732103 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:22.734965 (+ 2862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.736262 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.739192 (+ 2930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.739937 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.740045 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:22.751660 (+ 11615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:22.751667 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:22.751672 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:22.751713 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:22.754907 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:22.755802 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:22.755808 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:22.755812 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:22.755960 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:22.756431 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:22.756434 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:22.756437 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:22.756805 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:22.757045 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:22.757425 (+ 380us) server_negotiation.cc:299] Negotiation successful
0809 04:44:22.757632 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5362,"thread_start_us":5245,"threads_started":1}
I20260809 04:44:22.763585 21075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:22.725978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40018 (local address 127.12.234.61:41005)
0809 04:44:22.728523 (+ 2545us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:22.728531 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:22.741903 (+ 13372us) server_negotiation.cc:407] Connection header received
0809 04:44:22.745455 (+ 3552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:22.745463 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:22.745552 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:22.745703 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:22.748082 (+ 2379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.749455 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.751661 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.752515 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.752629 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:22.756928 (+ 4299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:22.756936 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:22.756943 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:22.756988 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:22.760406 (+ 3418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:22.761162 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:22.761169 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:22.761175 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:22.761324 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:22.761729 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:22.761734 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:22.761737 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:22.762068 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:22.762344 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:22.762922 (+ 578us) server_negotiation.cc:299] Negotiation successful
0809 04:44:22.763117 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2396,"thread_start_us":2287,"threads_started":1}
I20260809 04:44:22.762341 19716 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 04:44:22.769658 21066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:22.731890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37674 (local address 127.12.234.60:41485)
0809 04:44:22.740913 (+ 9023us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:22.740920 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:22.754624 (+ 13704us) server_negotiation.cc:407] Connection header received
0809 04:44:22.754870 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:22.754878 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:22.755020 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:22.755110 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:22.758352 (+ 3242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.759377 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.761236 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.761724 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.761865 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:22.764581 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:22.764589 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:22.764594 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:22.764623 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:22.767041 (+ 2418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:22.767780 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:22.767784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:22.767786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:22.767921 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:22.768335 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:22.768338 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:22.768339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:22.768566 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:22.768722 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:22.769148 (+ 426us) server_negotiation.cc:299] Negotiation successful
0809 04:44:22.769322 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8898,"thread_start_us":24677,"threads_started":1}
W20260809 04:44:22.770432 19716 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:44:22.801566 21094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:22.788630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:34979 (local address 127.0.0.1:39200)
0809 04:44:22.791961 (+ 3331us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:22.791982 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:22.792146 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:22.793205 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:22.793213 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:22.793250 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:22.793748 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:22.793763 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.795765 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.795770 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:22.798343 (+ 2573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:22.798363 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.800606 (+ 2243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.800614 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:22.800827 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:22.800865 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 04:44:22.800930 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 04:44:22.801083 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3194,"thread_start_us":12053,"threads_started":1}
I20260809 04:44:22.802822 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:22.789171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39200 (local address 127.12.234.2:34979)
0809 04:44:22.790475 (+ 1304us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:22.790484 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:22.792274 (+ 1790us) server_negotiation.cc:407] Connection header received
0809 04:44:22.792426 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:22.792437 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:22.792712 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:22.793225 (+ 513us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:22.793947 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.795570 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.798536 (+ 2966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.800323 (+ 1787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.801893 (+ 1570us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:22.802167 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:22.802257 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 04:44:22.802486 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1108,"thread_start_us":1280,"threads_started":1}
I20260809 04:44:22.806777 20839 tablet_service.cc:1467] Processing CreateTablet for tablet 67df49418260460690094fce6414dd4b (DEFAULT_TABLE table=test-table [id=9dc5d733e1274c1aa3942d38c28bb93d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:22.808281 20839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67df49418260460690094fce6414dd4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:22.810295 21093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:22.791405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:46629 (local address 127.0.0.1:54734)
0809 04:44:22.800957 (+ 9552us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:22.800973 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:22.801121 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:22.802348 (+ 1227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:22.802352 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:22.802372 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:22.802609 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:22.802616 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.805989 (+ 3373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.805995 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:22.807796 (+ 1801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:22.807810 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.809411 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.809418 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:22.809596 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:22.809643 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 04:44:22.809746 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 04:44:22.809900 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9418,"mutex_wait_us":248,"thread_start_us":282,"threads_started":1}
I20260809 04:44:22.810685 21095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:22.791634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54734 (local address 127.12.234.1:46629)
0809 04:44:22.800559 (+ 8925us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:22.800571 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:22.801124 (+ 553us) server_negotiation.cc:407] Connection header received
0809 04:44:22.801247 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:22.801257 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:22.801493 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:22.802170 (+ 677us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:22.802752 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.803868 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.807975 (+ 4107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.809199 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.809359 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:22.809579 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:22.809829 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 04:44:22.810027 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8734,"thread_start_us":8603,"threads_started":1}
I20260809 04:44:22.812182 21089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:22.788135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:42793 (local address 127.0.0.1:41120)
0809 04:44:22.789826 (+ 1691us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:22.789851 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:22.790001 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:22.804042 (+ 14041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:22.804049 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:22.804107 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:22.804506 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:22.804516 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.805920 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.805925 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:22.808065 (+ 2140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:22.808108 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.811278 (+ 3170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.811284 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:22.811474 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:22.811506 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 04:44:22.811590 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 04:44:22.811717 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1520,"mutex_wait_us":3,"thread_start_us":1357,"threads_started":1}
I20260809 04:44:22.812182 21090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:22.788405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41120 (local address 127.12.234.3:42793)
0809 04:44:22.803180 (+ 14775us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:22.803188 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:22.803222 (+ 34us) server_negotiation.cc:407] Connection header received
0809 04:44:22.803313 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:22.803320 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:22.803514 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:22.803832 (+ 318us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:22.804761 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.805771 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.809680 (+ 3909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:22.811083 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:22.811225 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:22.811426 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:22.811655 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 04:44:22.811800 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14598,"thread_start_us":15763,"threads_started":1}
I20260809 04:44:22.814414 20684 tablet_service.cc:1467] Processing CreateTablet for tablet 67df49418260460690094fce6414dd4b (DEFAULT_TABLE table=test-table [id=9dc5d733e1274c1aa3942d38c28bb93d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:22.815171 20988 tablet_service.cc:1467] Processing CreateTablet for tablet 67df49418260460690094fce6414dd4b (DEFAULT_TABLE table=test-table [id=9dc5d733e1274c1aa3942d38c28bb93d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:22.815778 20684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67df49418260460690094fce6414dd4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:22.816283 20988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67df49418260460690094fce6414dd4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:22.825076 21097 tablet_bootstrap.cc:492] T 67df49418260460690094fce6414dd4b P de0ed2dbaf5d438eae07171c965a3142: Bootstrap starting.
I20260809 04:44:22.828493 21098 tablet_bootstrap.cc:492] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0: Bootstrap starting.
I20260809 04:44:22.829276 21097 tablet_bootstrap.cc:654] T 67df49418260460690094fce6414dd4b P de0ed2dbaf5d438eae07171c965a3142: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:22.830351 21097 log.cc:824] T 67df49418260460690094fce6414dd4b P de0ed2dbaf5d438eae07171c965a3142: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:22.832679 21098 tablet_bootstrap.cc:654] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:22.833911 21098 log.cc:824] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:22.834565 21097 tablet_bootstrap.cc:492] T 67df49418260460690094fce6414dd4b P de0ed2dbaf5d438eae07171c965a3142: No bootstrap required, opened a new log
I20260809 04:44:22.834817 21097 ts_tablet_manager.cc:1397] T 67df49418260460690094fce6414dd4b P de0ed2dbaf5d438eae07171c965a3142: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 04:44:22.836381 21099 tablet_bootstrap.cc:492] T 67df49418260460690094fce6414dd4b P 2833ba21f588473f8daa7d439c8d7518: Bootstrap starting.
I20260809 04:44:22.836428 21098 tablet_bootstrap.cc:492] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0: No bootstrap required, opened a new log
I20260809 04:44:22.836670 21098 ts_tablet_manager.cc:1397] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260809 04:44:22.839564 21099 tablet_bootstrap.cc:654] T 67df49418260460690094fce6414dd4b P 2833ba21f588473f8daa7d439c8d7518: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:22.841938 21099 log.cc:824] T 67df49418260460690094fce6414dd4b P 2833ba21f588473f8daa7d439c8d7518: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:22.844911 21097 raft_consensus.cc:348] T 67df49418260460690094fce6414dd4b P de0ed2dbaf5d438eae07171c965a3142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f385b7e1143a5a32f5862960236c0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42793 } } peers { permanent_uuid: "de0ed2dbaf5d438eae07171c965a3142" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 34979 } } peers { permanent_uuid: "2833ba21f588473f8daa7d439c8d7518" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 46629 } }
I20260809 04:44:22.845446 21097 raft_consensus.cc:374] T 67df49418260460690094fce6414dd4b P de0ed2dbaf5d438eae07171c965a3142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:22.845571 21097 raft_consensus.cc:729] T 67df49418260460690094fce6414dd4b P de0ed2dbaf5d438eae07171c965a3142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de0ed2dbaf5d438eae07171c965a3142, State: Initialized, Role: FOLLOWER
I20260809 04:44:22.845825 21098 raft_consensus.cc:348] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f385b7e1143a5a32f5862960236c0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42793 } } peers { permanent_uuid: "de0ed2dbaf5d438eae07171c965a3142" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 34979 } } peers { permanent_uuid: "2833ba21f588473f8daa7d439c8d7518" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 46629 } }
I20260809 04:44:22.846189 21098 raft_consensus.cc:374] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:22.846287 21098 raft_consensus.cc:729] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 571f385b7e1143a5a32f5862960236c0, State: Initialized, Role: FOLLOWER
I20260809 04:44:22.846442 21097 consensus_queue.cc:260] T 67df49418260460690094fce6414dd4b P de0ed2dbaf5d438eae07171c965a3142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f385b7e1143a5a32f5862960236c0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42793 } } peers { permanent_uuid: "de0ed2dbaf5d438eae07171c965a3142" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 34979 } } peers { permanent_uuid: "2833ba21f588473f8daa7d439c8d7518" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 46629 } }
I20260809 04:44:22.846949 21098 consensus_queue.cc:260] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f385b7e1143a5a32f5862960236c0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42793 } } peers { permanent_uuid: "de0ed2dbaf5d438eae07171c965a3142" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 34979 } } peers { permanent_uuid: "2833ba21f588473f8daa7d439c8d7518" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 46629 } }
I20260809 04:44:22.855238 21098 ts_tablet_manager.cc:1428] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0: Time spent starting tablet: real 0.018s user 0.011s sys 0.007s
I20260809 04:44:22.858919 21058 heartbeater.cc:503] Master 127.12.234.62:44473 was elected leader, sending a full tablet report...
I20260809 04:44:22.861636 21099 tablet_bootstrap.cc:492] T 67df49418260460690094fce6414dd4b P 2833ba21f588473f8daa7d439c8d7518: No bootstrap required, opened a new log
I20260809 04:44:22.861937 21099 ts_tablet_manager.cc:1397] T 67df49418260460690094fce6414dd4b P 2833ba21f588473f8daa7d439c8d7518: Time spent bootstrapping tablet: real 0.029s user 0.006s sys 0.015s
I20260809 04:44:22.865577 21097 ts_tablet_manager.cc:1428] T 67df49418260460690094fce6414dd4b P de0ed2dbaf5d438eae07171c965a3142: Time spent starting tablet: real 0.031s user 0.016s sys 0.015s
I20260809 04:44:22.866173 20910 heartbeater.cc:503] Master 127.12.234.62:44473 was elected leader, sending a full tablet report...
W20260809 04:44:22.872243 20912 tablet.cc:2369] T 67df49418260460690094fce6414dd4b P de0ed2dbaf5d438eae07171c965a3142: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:44:22.872915 21099 raft_consensus.cc:348] T 67df49418260460690094fce6414dd4b P 2833ba21f588473f8daa7d439c8d7518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f385b7e1143a5a32f5862960236c0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42793 } } peers { permanent_uuid: "de0ed2dbaf5d438eae07171c965a3142" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 34979 } } peers { permanent_uuid: "2833ba21f588473f8daa7d439c8d7518" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 46629 } }
I20260809 04:44:22.873318 21099 raft_consensus.cc:374] T 67df49418260460690094fce6414dd4b P 2833ba21f588473f8daa7d439c8d7518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:22.873423 21099 raft_consensus.cc:729] T 67df49418260460690094fce6414dd4b P 2833ba21f588473f8daa7d439c8d7518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2833ba21f588473f8daa7d439c8d7518, State: Initialized, Role: FOLLOWER
I20260809 04:44:22.873951 21099 consensus_queue.cc:260] T 67df49418260460690094fce6414dd4b P 2833ba21f588473f8daa7d439c8d7518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f385b7e1143a5a32f5862960236c0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42793 } } peers { permanent_uuid: "de0ed2dbaf5d438eae07171c965a3142" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 34979 } } peers { permanent_uuid: "2833ba21f588473f8daa7d439c8d7518" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 46629 } }
I20260809 04:44:22.875468 21099 ts_tablet_manager.cc:1428] T 67df49418260460690094fce6414dd4b P 2833ba21f588473f8daa7d439c8d7518: Time spent starting tablet: real 0.013s user 0.011s sys 0.002s
W20260809 04:44:22.882823 20758 tablet.cc:2369] T 67df49418260460690094fce6414dd4b P 2833ba21f588473f8daa7d439c8d7518: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:44:22.957443 21060 tablet.cc:2369] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:44:23.003434 21104 raft_consensus.cc:482] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:44:23.003749 21104 raft_consensus.cc:504] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f385b7e1143a5a32f5862960236c0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42793 } } peers { permanent_uuid: "de0ed2dbaf5d438eae07171c965a3142" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 34979 } } peers { permanent_uuid: "2833ba21f588473f8daa7d439c8d7518" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 46629 } }
I20260809 04:44:23.005496 21104 leader_election.cc:288] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de0ed2dbaf5d438eae07171c965a3142 (127.12.234.2:34979), 2833ba21f588473f8daa7d439c8d7518 (127.12.234.1:46629)
I20260809 04:44:23.014176 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:23.006152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:45681 (local address 127.12.234.2:34979)
0809 04:44:23.006389 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:23.006394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:23.006645 (+ 251us) server_negotiation.cc:407] Connection header received
0809 04:44:23.006890 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:23.006895 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:23.006955 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:23.007078 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:23.007477 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:23.008284 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:23.011141 (+ 2857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:23.012306 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:23.012486 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:23.012552 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:23.013668 (+ 1116us) server_negotiation.cc:299] Negotiation successful
0809 04:44:23.013867 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260809 04:44:23.013489 21062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:23.005728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:34979 (local address 127.12.234.3:45681)
0809 04:44:23.006540 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:23.006558 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:23.006669 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:23.007099 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:23.007103 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:23.007136 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:23.007321 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:23.007330 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:23.008437 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:23.008446 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:23.010964 (+ 2518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:23.010974 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:23.012520 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:23.012528 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:23.012731 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:23.012785 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 04:44:23.012902 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 04:44:23.013051 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"mutex_wait_us":709}
I20260809 04:44:23.015249 21095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:23.005954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:35855 (local address 127.12.234.1:46629)
0809 04:44:23.006140 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:23.006146 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:23.006167 (+ 21us) server_negotiation.cc:407] Connection header received
0809 04:44:23.006448 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:23.006455 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:23.006545 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:23.006647 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:23.007203 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:23.008400 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:23.012624 (+ 4224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:23.013763 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:23.013899 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:23.013960 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:23.014874 (+ 914us) server_negotiation.cc:299] Negotiation successful
0809 04:44:23.015006 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:44:23.015780 21064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:23.005728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:46629 (local address 127.12.234.3:35855)
0809 04:44:23.006063 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:23.006088 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:23.006220 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:23.006760 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:23.006766 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:23.006798 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:23.007008 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:23.007018 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:23.008519 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:23.008524 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:23.010957 (+ 2433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:23.010974 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:23.014093 (+ 3119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:23.014099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:23.014290 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:23.014321 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 04:44:23.014406 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 04:44:23.014527 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":201}
I20260809 04:44:23.016047 20861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67df49418260460690094fce6414dd4b" candidate_uuid: "571f385b7e1143a5a32f5862960236c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de0ed2dbaf5d438eae07171c965a3142" is_pre_election: true
I20260809 04:44:23.016813 20861 raft_consensus.cc:2454] T 67df49418260460690094fce6414dd4b P de0ed2dbaf5d438eae07171c965a3142 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 571f385b7e1143a5a32f5862960236c0 in term 0.
I20260809 04:44:23.017064 20705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67df49418260460690094fce6414dd4b" candidate_uuid: "571f385b7e1143a5a32f5862960236c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2833ba21f588473f8daa7d439c8d7518" is_pre_election: true
I20260809 04:44:23.017491 20705 raft_consensus.cc:2454] T 67df49418260460690094fce6414dd4b P 2833ba21f588473f8daa7d439c8d7518 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 571f385b7e1143a5a32f5862960236c0 in term 0.
I20260809 04:44:23.017570 20940 leader_election.cc:302] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 571f385b7e1143a5a32f5862960236c0, de0ed2dbaf5d438eae07171c965a3142; no voters:
I20260809 04:44:23.018126 21104 raft_consensus.cc:2781] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:44:23.018306 21104 raft_consensus.cc:482] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:44:23.018429 21104 raft_consensus.cc:3037] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:23.022276 21104 raft_consensus.cc:504] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f385b7e1143a5a32f5862960236c0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42793 } } peers { permanent_uuid: "de0ed2dbaf5d438eae07171c965a3142" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 34979 } } peers { permanent_uuid: "2833ba21f588473f8daa7d439c8d7518" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 46629 } }
I20260809 04:44:23.023111 21104 leader_election.cc:288] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0 [CANDIDATE]: Term 1 election: Requested vote from peers de0ed2dbaf5d438eae07171c965a3142 (127.12.234.2:34979), 2833ba21f588473f8daa7d439c8d7518 (127.12.234.1:46629)
I20260809 04:44:23.023567 20705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67df49418260460690094fce6414dd4b" candidate_uuid: "571f385b7e1143a5a32f5862960236c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2833ba21f588473f8daa7d439c8d7518"
I20260809 04:44:23.023612 20861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67df49418260460690094fce6414dd4b" candidate_uuid: "571f385b7e1143a5a32f5862960236c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de0ed2dbaf5d438eae07171c965a3142"
I20260809 04:44:23.023847 20705 raft_consensus.cc:3037] T 67df49418260460690094fce6414dd4b P 2833ba21f588473f8daa7d439c8d7518 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:23.023886 20861 raft_consensus.cc:3037] T 67df49418260460690094fce6414dd4b P de0ed2dbaf5d438eae07171c965a3142 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:23.026778 20705 raft_consensus.cc:2454] T 67df49418260460690094fce6414dd4b P 2833ba21f588473f8daa7d439c8d7518 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 571f385b7e1143a5a32f5862960236c0 in term 1.
I20260809 04:44:23.027202 20941 leader_election.cc:302] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2833ba21f588473f8daa7d439c8d7518, 571f385b7e1143a5a32f5862960236c0; no voters:
I20260809 04:44:23.027531 21104 raft_consensus.cc:2781] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:23.028352 21104 raft_consensus.cc:686] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0 [term 1 LEADER]: Becoming Leader. State: Replica: 571f385b7e1143a5a32f5862960236c0, State: Running, Role: LEADER
I20260809 04:44:23.029076 21104 consensus_queue.cc:237] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f385b7e1143a5a32f5862960236c0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42793 } } peers { permanent_uuid: "de0ed2dbaf5d438eae07171c965a3142" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 34979 } } peers { permanent_uuid: "2833ba21f588473f8daa7d439c8d7518" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 46629 } }
I20260809 04:44:23.029583 20861 raft_consensus.cc:2454] T 67df49418260460690094fce6414dd4b P de0ed2dbaf5d438eae07171c965a3142 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 571f385b7e1143a5a32f5862960236c0 in term 1.
I20260809 04:44:23.035888 19716 catalog_manager.cc:5495] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 571f385b7e1143a5a32f5862960236c0 (127.12.234.3). New cstate: current_term: 1 leader_uuid: "571f385b7e1143a5a32f5862960236c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f385b7e1143a5a32f5862960236c0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de0ed2dbaf5d438eae07171c965a3142" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 34979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2833ba21f588473f8daa7d439c8d7518" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 46629 } health_report { overall_health: UNKNOWN } } }
I20260809 04:44:23.107676 21090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:23.100749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41122 (local address 127.12.234.3:42793)
0809 04:44:23.101038 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:23.101047 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:23.101068 (+ 21us) server_negotiation.cc:407] Connection header received
0809 04:44:23.101352 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:23.101357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:23.101462 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:23.101625 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:44:23.102168 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:23.103123 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:23.104915 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:23.105518 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:23.105665 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:23.106393 (+ 728us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:44:23.106944 (+ 551us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:44:23.107115 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:23.107228 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 04:44:23.107326 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260809 04:44:23.119740 20861 raft_consensus.cc:1261] T 67df49418260460690094fce6414dd4b P de0ed2dbaf5d438eae07171c965a3142 [term 1 FOLLOWER]: Refusing update from remote peer 571f385b7e1143a5a32f5862960236c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:23.119683 20705 raft_consensus.cc:1261] T 67df49418260460690094fce6414dd4b P 2833ba21f588473f8daa7d439c8d7518 [term 1 FOLLOWER]: Refusing update from remote peer 571f385b7e1143a5a32f5862960236c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:23.120925 21106 consensus_queue.cc:1035] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de0ed2dbaf5d438eae07171c965a3142" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 34979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:23.121567 21104 consensus_queue.cc:1035] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2833ba21f588473f8daa7d439c8d7518" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 46629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:23.141084 21109 mvcc.cc:205] Tried to move back new op lower bound from 7316482716124209152 to 7316482715773603840. Current Snapshot: MvccSnapshot[applied={T|T < 7316482716124209152}]
I20260809 04:44:23.164340 21112 mvcc.cc:205] Tried to move back new op lower bound from 7316482716124209152 to 7316482715773603840. Current Snapshot: MvccSnapshot[applied={T|T < 7316482716124209152}]
I20260809 04:44:23.177727 19716 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:23.178129 19716 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:23.187783 19716 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ec11c8eec1744553802f8652681fe77a: Sending DeleteTablet for 3 replicas of tablet 67df49418260460690094fce6414dd4b
I20260809 04:44:23.189183 20839 tablet_service.cc:1514] Processing DeleteTablet for tablet 67df49418260460690094fce6414dd4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:23 UTC) from {username='oryx', principal='oryx/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:39200
I20260809 04:44:23.189359 20684 tablet_service.cc:1514] Processing DeleteTablet for tablet 67df49418260460690094fce6414dd4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:23 UTC) from {username='oryx', principal='oryx/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:54734
I20260809 04:44:23.189359 20988 tablet_service.cc:1514] Processing DeleteTablet for tablet 67df49418260460690094fce6414dd4b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:23 UTC) from {username='oryx', principal='oryx/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:41120
I20260809 04:44:23.201603 21120 tablet_replica.cc:330] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0: stopping tablet replica
I20260809 04:44:23.201910 21119 tablet_replica.cc:330] T 67df49418260460690094fce6414dd4b P de0ed2dbaf5d438eae07171c965a3142: stopping tablet replica
I20260809 04:44:23.202178 21120 raft_consensus.cc:2229] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:44:23.202405 21119 raft_consensus.cc:2229] T 67df49418260460690094fce6414dd4b P de0ed2dbaf5d438eae07171c965a3142 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:44:23.203271 21119 raft_consensus.cc:2258] T 67df49418260460690094fce6414dd4b P de0ed2dbaf5d438eae07171c965a3142 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:23.205797 21119 ts_tablet_manager.cc:1905] T 67df49418260460690094fce6414dd4b P de0ed2dbaf5d438eae07171c965a3142: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:23.206200 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 19681
I20260809 04:44:23.208595 19747 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:23.208635 21120 raft_consensus.cc:2258] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:23.213798 21120 ts_tablet_manager.cc:1905] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:23.213946 21121 tablet_replica.cc:330] T 67df49418260460690094fce6414dd4b P 2833ba21f588473f8daa7d439c8d7518: stopping tablet replica
I20260809 04:44:23.214449 21121 raft_consensus.cc:2229] T 67df49418260460690094fce6414dd4b P 2833ba21f588473f8daa7d439c8d7518 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:44:23.215713 21121 raft_consensus.cc:2258] T 67df49418260460690094fce6414dd4b P 2833ba21f588473f8daa7d439c8d7518 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:23.219187 21121 ts_tablet_manager.cc:1905] T 67df49418260460690094fce6414dd4b P 2833ba21f588473f8daa7d439c8d7518: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:23.230970 21119 ts_tablet_manager.cc:1918] T 67df49418260460690094fce6414dd4b P de0ed2dbaf5d438eae07171c965a3142: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:23.231297 21119 log.cc:1196] T 67df49418260460690094fce6414dd4b P de0ed2dbaf5d438eae07171c965a3142: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/wal/wals/67df49418260460690094fce6414dd4b
I20260809 04:44:23.233875 21075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:23.194924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40022 (local address 127.12.234.61:41005)
0809 04:44:23.195088 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:23.195093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:23.206637 (+ 11544us) server_negotiation.cc:407] Connection header received
0809 04:44:23.206765 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:23.206770 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:23.206840 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:23.207093 (+ 253us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:23.211942 (+ 4849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:23.213381 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:23.216697 (+ 3316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:23.217393 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:23.217565 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:23.224106 (+ 6541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:23.224115 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:23.224122 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:23.224159 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:23.229676 (+ 5517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:23.230759 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:23.230765 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:23.230767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:23.231052 (+ 285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:23.231691 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:23.231699 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:23.231704 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:23.232289 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:23.232591 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:23.233106 (+ 515us) server_negotiation.cc:299] Negotiation successful
0809 04:44:23.233371 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 04:44:23.235512 21119 ts_tablet_manager.cc:1939] T 67df49418260460690094fce6414dd4b P de0ed2dbaf5d438eae07171c965a3142: Deleting consensus metadata
I20260809 04:44:23.239555 21120 ts_tablet_manager.cc:1918] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:23.241345 21121 ts_tablet_manager.cc:1918] T 67df49418260460690094fce6414dd4b P 2833ba21f588473f8daa7d439c8d7518: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:23.241707 21121 log.cc:1196] T 67df49418260460690094fce6414dd4b P 2833ba21f588473f8daa7d439c8d7518: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/wal/wals/67df49418260460690094fce6414dd4b
I20260809 04:44:23.242244 21120 log.cc:1196] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/wal/wals/67df49418260460690094fce6414dd4b
I20260809 04:44:23.243364 21120 ts_tablet_manager.cc:1939] T 67df49418260460690094fce6414dd4b P 571f385b7e1143a5a32f5862960236c0: Deleting consensus metadata
I20260809 04:44:23.244211 21121 ts_tablet_manager.cc:1939] T 67df49418260460690094fce6414dd4b P 2833ba21f588473f8daa7d439c8d7518: Deleting consensus metadata
I20260809 04:44:23.258522 21066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:23.205137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37690 (local address 127.12.234.60:41485)
0809 04:44:23.205297 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:23.205302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:23.233943 (+ 28641us) server_negotiation.cc:407] Connection header received
0809 04:44:23.235067 (+ 1124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:23.235072 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:23.235193 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:23.235320 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:23.238129 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:23.239561 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:23.242541 (+ 2980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:23.243583 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:23.243737 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:23.248328 (+ 4591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:23.248337 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:23.248354 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:23.248401 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:23.253971 (+ 5570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:23.255266 (+ 1295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:23.255272 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:23.255275 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:23.255467 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:23.256151 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:23.256157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:23.256160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:23.256587 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:23.256923 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:23.257567 (+ 644us) server_negotiation.cc:299] Negotiation successful
0809 04:44:23.257827 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 04:44:23.408521 19700 catalog_manager.cc:4841] TS 2833ba21f588473f8daa7d439c8d7518 (127.12.234.1:46629): tablet 67df49418260460690094fce6414dd4b (table test-table [id=9dc5d733e1274c1aa3942d38c28bb93d]) successfully deleted
I20260809 04:44:23.409930 19701 catalog_manager.cc:4841] TS 571f385b7e1143a5a32f5862960236c0 (127.12.234.3:42793): tablet 67df49418260460690094fce6414dd4b (table test-table [id=9dc5d733e1274c1aa3942d38c28bb93d]) successfully deleted
I20260809 04:44:23.410770 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 19681
I20260809 04:44:23.462235 21066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:23.444377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37700 (local address 127.12.234.60:41485)
0809 04:44:23.444585 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:23.444591 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:23.444609 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:44:23.444693 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:23.444698 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:23.444770 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:23.444884 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:23.447286 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:23.448421 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:23.450327 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:23.450820 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:23.450979 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:23.455574 (+ 4595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:23.455580 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:23.455584 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:23.455631 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:23.458715 (+ 3084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:23.459804 (+ 1089us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:23.459809 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:23.459813 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:23.460062 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:23.460526 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:23.460533 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:23.460539 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:23.460918 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:23.461145 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:23.461631 (+ 486us) server_negotiation.cc:299] Negotiation successful
0809 04:44:23.461863 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 04:44:23.462256 21075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:23.442937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40034 (local address 127.12.234.61:41005)
0809 04:44:23.443167 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:23.443173 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:23.443812 (+ 639us) server_negotiation.cc:407] Connection header received
0809 04:44:23.444278 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:23.444288 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:23.444387 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:23.444489 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:23.446296 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:23.447066 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:23.448900 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:23.449704 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:23.449807 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:23.454217 (+ 4410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:23.454254 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:23.454263 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:23.454453 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:23.458708 (+ 4255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:23.459949 (+ 1241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:23.459954 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:23.459957 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:23.460117 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:23.460571 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:23.460578 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:23.460583 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:23.460917 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:23.461134 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:23.461717 (+ 583us) server_negotiation.cc:299] Negotiation successful
0809 04:44:23.461926 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 04:44:24.155823 21131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.155008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.12.234.1:34817)
0809 04:44:24.155533 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:24.155645 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":304,"threads_started":1}
W20260809 04:44:24.156451 20756 heartbeater.cc:644] Failed to heartbeat to 127.12.234.62:44473 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
I20260809 04:44:24.166289 21132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.164677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.12.234.3:56775)
0809 04:44:24.165986 (+ 1309us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:24.166100 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1180,"thread_start_us":1095,"threads_started":1}
W20260809 04:44:24.166673 21058 heartbeater.cc:644] Failed to heartbeat to 127.12.234.62:44473 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
I20260809 04:44:24.168943 21133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.168316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.12.234.2:53999)
0809 04:44:24.168694 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:24.168794 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":187,"threads_started":1}
W20260809 04:44:24.169327 20910 heartbeater.cc:644] Failed to heartbeat to 127.12.234.62:44473 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
I20260809 04:44:24.687160 21134 raft_consensus.cc:482] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec11c8eec1744553802f8652681fe77a)
I20260809 04:44:24.687513 21134 raft_consensus.cc:504] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: false } }
I20260809 04:44:24.687796 21135 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ec11c8eec1744553802f8652681fe77a)
I20260809 04:44:24.688113 21135 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: false } }
I20260809 04:44:24.689299 21134 leader_election.cc:288] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec11c8eec1744553802f8652681fe77a (127.12.234.62:44473), b4a48d1bc3cb43ef90c0916f666eec83 (127.12.234.61:41005)
I20260809 04:44:24.690729 21135 leader_election.cc:288] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec11c8eec1744553802f8652681fe77a (127.12.234.62:44473), f12602d2831a44c397103d20328e6b17 (127.12.234.60:41485)
I20260809 04:44:24.692176 21137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.689502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.0.0.1:57528)
0809 04:44:24.691449 (+ 1947us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:24.691748 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1767,"mutex_wait_us":20,"thread_start_us":1573,"threads_started":1}
I20260809 04:44:24.694130 21138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.689949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.0.0.1:57538)
0809 04:44:24.693591 (+ 3642us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:24.693830 (+ 239us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3510,"thread_start_us":2624,"threads_started":1}
W20260809 04:44:24.693984 20541 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
W20260809 04:44:24.695278 20340 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
I20260809 04:44:24.700237 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.689756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:41485 (local address 127.0.0.1:37716)
0809 04:44:24.692904 (+ 3148us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:24.692927 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:24.693074 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:24.694286 (+ 1212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:24.694293 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:24.694321 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:24.694638 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:24.694647 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.696321 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.696328 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:24.697750 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:24.697762 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.699374 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.699380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:24.699559 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:24.699584 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 04:44:24.699697 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 04:44:24.699825 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2987,"thread_start_us":3570,"threads_started":1}
I20260809 04:44:24.700299 21140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.691443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37716 (local address 127.12.234.60:41485)
0809 04:44:24.693922 (+ 2479us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:24.693927 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:24.693948 (+ 21us) server_negotiation.cc:407] Connection header received
0809 04:44:24.694016 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:24.694021 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:24.694113 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:24.694221 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:24.694830 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.696212 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.697919 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.699233 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.699350 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:24.699441 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:24.699733 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 04:44:24.699864 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2339,"thread_start_us":2219,"threads_started":1}
I20260809 04:44:24.701560 20570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f12602d2831a44c397103d20328e6b17" is_pre_election: true
I20260809 04:44:24.701850 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.701441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.0.0.1:57542)
0809 04:44:24.701605 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:24.701705 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 04:44:24.701937 20570 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4a48d1bc3cb43ef90c0916f666eec83 in term 1.
W20260809 04:44:24.702021 20340 leader_election.cc:334] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ec11c8eec1744553802f8652681fe77a (127.12.234.62:44473): Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
I20260809 04:44:24.702767 21137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.702384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.0.0.1:57556)
0809 04:44:24.702538 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:24.702618 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 04:44:24.702745 20341 leader_election.cc:302] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4a48d1bc3cb43ef90c0916f666eec83, f12602d2831a44c397103d20328e6b17; no voters: ec11c8eec1744553802f8652681fe77a
I20260809 04:44:24.703116 21135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 04:44:24.703271 21135 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ec11c8eec1744553802f8652681fe77a)
I20260809 04:44:24.703387 21135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [term 1 FOLLOWER]: Advancing to term 2
W20260809 04:44:24.703392 20541 leader_election.cc:334] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ec11c8eec1744553802f8652681fe77a (127.12.234.62:44473): Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
I20260809 04:44:24.705487 21136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.689490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:41005 (local address 127.0.0.1:40046)
0809 04:44:24.690301 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:24.690327 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:24.690484 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:24.695582 (+ 5098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:24.695587 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:24.695613 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:24.696014 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:24.696026 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.700723 (+ 4697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.700731 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:24.702860 (+ 2129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:24.702877 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.704705 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.704711 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:24.704916 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:24.704952 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 04:44:24.705047 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 04:44:24.705171 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":1102,"threads_started":1}
I20260809 04:44:24.705598 21141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.692897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40046 (local address 127.12.234.61:41005)
0809 04:44:24.695256 (+ 2359us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:24.695266 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:24.695285 (+ 19us) server_negotiation.cc:407] Connection header received
0809 04:44:24.695338 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:24.695343 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:24.695416 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:24.696199 (+ 783us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:24.698134 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.700524 (+ 2390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.703024 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.704517 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.704656 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:24.704771 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:24.705092 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 04:44:24.705236 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2237,"thread_start_us":2077,"threads_started":1}
I20260809 04:44:24.706354 20367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f12602d2831a44c397103d20328e6b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" is_pre_election: true
I20260809 04:44:24.706491 21135 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: false } }
I20260809 04:44:24.707038 20367 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f12602d2831a44c397103d20328e6b17 in current term 2: Already voted for candidate b4a48d1bc3cb43ef90c0916f666eec83 in this term.
I20260809 04:44:24.707268 21135 leader_election.cc:288] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [CANDIDATE]: Term 2 election: Requested vote from peers ec11c8eec1744553802f8652681fe77a (127.12.234.62:44473), f12602d2831a44c397103d20328e6b17 (127.12.234.60:41485)
I20260809 04:44:24.707751 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.707338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.0.0.1:57558)
0809 04:44:24.707524 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:24.707612 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 04:44:24.707837 20542 leader_election.cc:302] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f12602d2831a44c397103d20328e6b17; no voters: b4a48d1bc3cb43ef90c0916f666eec83, ec11c8eec1744553802f8652681fe77a
I20260809 04:44:24.707898 20570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f12602d2831a44c397103d20328e6b17"
I20260809 04:44:24.708170 20570 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [term 1 FOLLOWER]: Advancing to term 2
I20260809 04:44:24.708787 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.708457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.0.0.1:57566)
0809 04:44:24.708604 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:24.708681 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 04:44:24.709028 20340 leader_election.cc:334] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ec11c8eec1744553802f8652681fe77a (127.12.234.62:44473): Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
I20260809 04:44:24.710494 20570 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4a48d1bc3cb43ef90c0916f666eec83 in term 2.
I20260809 04:44:24.710687 21134 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 04:44:24.710922 20341 leader_election.cc:302] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b4a48d1bc3cb43ef90c0916f666eec83, f12602d2831a44c397103d20328e6b17; no voters: ec11c8eec1744553802f8652681fe77a
I20260809 04:44:24.711242 21135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 04:44:24.711688 21135 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [term 2 LEADER]: Becoming Leader. State: Replica: b4a48d1bc3cb43ef90c0916f666eec83, State: Running, Role: LEADER
I20260809 04:44:24.712067 21135 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: false } }
I20260809 04:44:24.714089 21144 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4a48d1bc3cb43ef90c0916f666eec83. Latest consensus state: current_term: 2 leader_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: false } } }
I20260809 04:44:24.714541 21144 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:24.715193 21146 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:24.719224 21146 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:24.720067 21146 catalog_manager.cc:1259] Loaded cluster ID: 31871136da804a259b763c92b6066c36
I20260809 04:44:24.720221 21146 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:24.721391 21146 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:24.722231 21146 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83: Loaded TSK: 0
I20260809 04:44:24.723126 21146 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:44:24.769313 20356 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40034:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:44:24.776644 20356 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:44:24.780994 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.780533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.0.0.1:57594)
0809 04:44:24.780730 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:24.780805 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 04:44:24.780968 20570 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [term 2 FOLLOWER]: Refusing update from remote peer b4a48d1bc3cb43ef90c0916f666eec83: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260809 04:44:24.781893 21144 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 04:44:24.782496 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.782098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.0.0.1:57610)
0809 04:44:24.782278 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:24.782360 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 04:44:24.782701 20340 consensus_peers.cc:487] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 -> Peer ec11c8eec1744553802f8652681fe77a (127.12.234.62:44473): Couldn't send request to peer ec11c8eec1744553802f8652681fe77a. Status: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 04:44:24.785300 21134 sys_catalog.cc:455] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4a48d1bc3cb43ef90c0916f666eec83. Latest consensus state: current_term: 2 leader_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: false } } }
I20260809 04:44:24.785573 21134 sys_catalog.cc:458] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:44:24.787230 21144 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: false } } }
I20260809 04:44:24.787374 21148 mvcc.cc:205] Tried to move back new op lower bound from 7316482722935767040 to 7316482722664415232. Current Snapshot: MvccSnapshot[applied={T|T < 7316482722935767040}]
I20260809 04:44:24.787550 21144 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:24.788023 21134 sys_catalog.cc:455] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ec11c8eec1744553802f8652681fe77a" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44473 } } peers { permanent_uuid: "b4a48d1bc3cb43ef90c0916f666eec83" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 41005 } attrs { promote: false } } peers { permanent_uuid: "f12602d2831a44c397103d20328e6b17" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 41485 } attrs { promote: false } } }
I20260809 04:44:24.788321 21134 sys_catalog.cc:458] T 00000000000000000000000000000000 P f12602d2831a44c397103d20328e6b17 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:44:24.806898 21154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.797415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41138 (local address 127.12.234.3:42793)
0809 04:44:24.798583 (+ 1168us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:24.798590 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:24.798613 (+ 23us) server_negotiation.cc:407] Connection header received
0809 04:44:24.798694 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:24.798700 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:24.798815 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:24.799551 (+ 736us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:24.799817 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.801447 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.804583 (+ 3136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.805659 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.805818 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:24.805977 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:24.806346 (+ 369us) server_negotiation.cc:299] Negotiation successful
0809 04:44:24.806491 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1034,"thread_start_us":896,"threads_started":1}
I20260809 04:44:24.806921 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.797169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:42793 (local address 127.0.0.1:41138)
0809 04:44:24.797450 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:24.797469 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:24.797664 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:24.799015 (+ 1351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:24.799023 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:24.799061 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:24.799375 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:24.799392 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.801654 (+ 2262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.801663 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:24.804355 (+ 2692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:24.804378 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.805867 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.805875 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:24.806128 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:24.806183 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 04:44:24.806297 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 04:44:24.806483 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176}
I20260809 04:44:24.808166 20988 tablet_service.cc:1467] Processing CreateTablet for tablet d5dc2dd7b87b4abb8ac8da1cb390ee8f (DEFAULT_TABLE table=test-table [id=9b869a5f3d8d4cd7ad069eec0c03750c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:24.808799 20988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5dc2dd7b87b4abb8ac8da1cb390ee8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:24.811796 21157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.798222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:46629 (local address 127.0.0.1:54744)
0809 04:44:24.800192 (+ 1970us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:24.800220 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:24.800354 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:24.804919 (+ 4565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:24.804927 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:24.804964 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:24.805388 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:24.805401 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.806757 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.806763 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:24.809649 (+ 2886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:24.809664 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.810881 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.810888 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:24.811063 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:24.811088 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 04:44:24.811210 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 04:44:24.811392 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1792,"thread_start_us":1576,"threads_started":1}
I20260809 04:44:24.811795 21158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.798691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54744 (local address 127.12.234.1:46629)
0809 04:44:24.804561 (+ 5870us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:24.804567 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:24.804588 (+ 21us) server_negotiation.cc:407] Connection header received
0809 04:44:24.804673 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:24.804680 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:24.804761 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:24.804880 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:24.805586 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.806579 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.809841 (+ 3262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.810648 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.810825 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:24.810954 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:24.811252 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 04:44:24.811427 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5705,"thread_start_us":5599,"threads_started":1}
I20260809 04:44:24.812942 20684 tablet_service.cc:1467] Processing CreateTablet for tablet d5dc2dd7b87b4abb8ac8da1cb390ee8f (DEFAULT_TABLE table=test-table [id=9b869a5f3d8d4cd7ad069eec0c03750c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:24.813503 20684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5dc2dd7b87b4abb8ac8da1cb390ee8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:24.815093 21159 tablet_bootstrap.cc:492] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 571f385b7e1143a5a32f5862960236c0: Bootstrap starting.
I20260809 04:44:24.817061 21159 tablet_bootstrap.cc:654] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 571f385b7e1143a5a32f5862960236c0: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:24.819949 21161 tablet_bootstrap.cc:492] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 2833ba21f588473f8daa7d439c8d7518: Bootstrap starting.
I20260809 04:44:24.820437 21159 tablet_bootstrap.cc:492] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 571f385b7e1143a5a32f5862960236c0: No bootstrap required, opened a new log
I20260809 04:44:24.820689 21159 ts_tablet_manager.cc:1397] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 571f385b7e1143a5a32f5862960236c0: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260809 04:44:24.821393 21138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.797742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:34979 (local address 127.0.0.1:39208)
0809 04:44:24.797898 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:24.797914 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:24.798015 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:24.814757 (+ 16742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:24.814764 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:24.814797 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:24.815072 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:24.815087 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.816663 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.816669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:24.818780 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:24.818801 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.820542 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.820549 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:24.820757 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:24.820798 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 04:44:24.820929 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 04:44:24.821107 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 04:44:24.821405 21156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.797861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39208 (local address 127.12.234.2:34979)
0809 04:44:24.810381 (+ 12520us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:24.810390 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:24.810410 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:44:24.810483 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:24.810489 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:24.810563 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:24.810667 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:24.815342 (+ 4675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.816494 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.818987 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.820292 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.820430 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:24.820544 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:24.820966 (+ 422us) server_negotiation.cc:299] Negotiation successful
0809 04:44:24.821129 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12390,"thread_start_us":12911,"threads_started":1}
I20260809 04:44:24.821745 21161 tablet_bootstrap.cc:654] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 2833ba21f588473f8daa7d439c8d7518: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:24.821653 21159 raft_consensus.cc:348] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 571f385b7e1143a5a32f5862960236c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f385b7e1143a5a32f5862960236c0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42793 } } peers { permanent_uuid: "de0ed2dbaf5d438eae07171c965a3142" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 34979 } } peers { permanent_uuid: "2833ba21f588473f8daa7d439c8d7518" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 46629 } }
I20260809 04:44:24.821877 21159 raft_consensus.cc:374] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 571f385b7e1143a5a32f5862960236c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:24.821998 21159 raft_consensus.cc:729] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 571f385b7e1143a5a32f5862960236c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 571f385b7e1143a5a32f5862960236c0, State: Initialized, Role: FOLLOWER
I20260809 04:44:24.822247 20839 tablet_service.cc:1467] Processing CreateTablet for tablet d5dc2dd7b87b4abb8ac8da1cb390ee8f (DEFAULT_TABLE table=test-table [id=9b869a5f3d8d4cd7ad069eec0c03750c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:24.822252 21159 consensus_queue.cc:260] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 571f385b7e1143a5a32f5862960236c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f385b7e1143a5a32f5862960236c0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42793 } } peers { permanent_uuid: "de0ed2dbaf5d438eae07171c965a3142" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 34979 } } peers { permanent_uuid: "2833ba21f588473f8daa7d439c8d7518" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 46629 } }
I20260809 04:44:24.822726 20839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5dc2dd7b87b4abb8ac8da1cb390ee8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:24.823228 21159 ts_tablet_manager.cc:1428] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 571f385b7e1143a5a32f5862960236c0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 04:44:24.826526 21161 tablet_bootstrap.cc:492] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 2833ba21f588473f8daa7d439c8d7518: No bootstrap required, opened a new log
I20260809 04:44:24.826717 21161 ts_tablet_manager.cc:1397] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 2833ba21f588473f8daa7d439c8d7518: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260809 04:44:24.827420 21161 raft_consensus.cc:348] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 2833ba21f588473f8daa7d439c8d7518 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f385b7e1143a5a32f5862960236c0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42793 } } peers { permanent_uuid: "de0ed2dbaf5d438eae07171c965a3142" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 34979 } } peers { permanent_uuid: "2833ba21f588473f8daa7d439c8d7518" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 46629 } }
I20260809 04:44:24.827664 21161 raft_consensus.cc:374] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 2833ba21f588473f8daa7d439c8d7518 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:24.827802 21161 raft_consensus.cc:729] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 2833ba21f588473f8daa7d439c8d7518 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2833ba21f588473f8daa7d439c8d7518, State: Initialized, Role: FOLLOWER
I20260809 04:44:24.828166 21059 heartbeater.cc:503] Master 127.12.234.61:41005 was elected leader, sending a full tablet report...
I20260809 04:44:24.828157 21161 consensus_queue.cc:260] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 2833ba21f588473f8daa7d439c8d7518 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f385b7e1143a5a32f5862960236c0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42793 } } peers { permanent_uuid: "de0ed2dbaf5d438eae07171c965a3142" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 34979 } } peers { permanent_uuid: "2833ba21f588473f8daa7d439c8d7518" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 46629 } }
I20260809 04:44:24.829137 21163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.823453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.12.234.3:40575)
0809 04:44:24.825707 (+ 2254us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:24.828841 (+ 3134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2111,"thread_start_us":1972,"threads_started":1}
I20260809 04:44:24.830384 21163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.829876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.12.234.3:51841)
0809 04:44:24.830098 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:24.830228 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260809 04:44:24.833036 21166 tablet_bootstrap.cc:492] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142: Bootstrap starting.
I20260809 04:44:24.833532 21161 ts_tablet_manager.cc:1428] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 2833ba21f588473f8daa7d439c8d7518: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260809 04:44:24.834900 21166 tablet_bootstrap.cc:654] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:24.835330 21167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.834250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.12.234.1:54895)
0809 04:44:24.834823 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:24.835040 (+ 217us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"threads_started":1}
I20260809 04:44:24.836117 20757 heartbeater.cc:503] Master 127.12.234.61:41005 was elected leader, sending a full tablet report...
I20260809 04:44:24.837124 21167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.836666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.12.234.1:42651)
0809 04:44:24.836898 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:24.836982 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260809 04:44:24.837637 21166 tablet_bootstrap.cc:492] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142: No bootstrap required, opened a new log
I20260809 04:44:24.837781 21166 ts_tablet_manager.cc:1397] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 04:44:24.838619 21166 raft_consensus.cc:348] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f385b7e1143a5a32f5862960236c0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42793 } } peers { permanent_uuid: "de0ed2dbaf5d438eae07171c965a3142" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 34979 } } peers { permanent_uuid: "2833ba21f588473f8daa7d439c8d7518" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 46629 } }
I20260809 04:44:24.838815 21166 raft_consensus.cc:374] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:24.838877 21166 raft_consensus.cc:729] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de0ed2dbaf5d438eae07171c965a3142, State: Initialized, Role: FOLLOWER
I20260809 04:44:24.839128 21166 consensus_queue.cc:260] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f385b7e1143a5a32f5862960236c0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42793 } } peers { permanent_uuid: "de0ed2dbaf5d438eae07171c965a3142" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 34979 } } peers { permanent_uuid: "2833ba21f588473f8daa7d439c8d7518" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 46629 } }
I20260809 04:44:24.839864 21166 ts_tablet_manager.cc:1428] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 04:44:24.842737 21170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.840579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.12.234.2:40357)
0809 04:44:24.842415 (+ 1836us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:24.842551 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1703,"thread_start_us":1537,"threads_started":1}
I20260809 04:44:24.954891 21169 raft_consensus.cc:482] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:44:24.955216 21169 raft_consensus.cc:504] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f385b7e1143a5a32f5862960236c0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42793 } } peers { permanent_uuid: "de0ed2dbaf5d438eae07171c965a3142" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 34979 } } peers { permanent_uuid: "2833ba21f588473f8daa7d439c8d7518" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 46629 } }
I20260809 04:44:24.957399 21169 leader_election.cc:288] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 571f385b7e1143a5a32f5862960236c0 (127.12.234.3:42793), 2833ba21f588473f8daa7d439c8d7518 (127.12.234.1:46629)
I20260809 04:44:24.963964 21154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.957698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:53247 (local address 127.12.234.3:42793)
0809 04:44:24.957946 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:24.957955 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:24.958732 (+ 777us) server_negotiation.cc:407] Connection header received
0809 04:44:24.958887 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:24.958893 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:24.958984 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:24.959065 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:24.959739 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.960517 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.962240 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.962990 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.963184 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:24.963267 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:24.963579 (+ 312us) server_negotiation.cc:299] Negotiation successful
0809 04:44:24.963704 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 04:44:24.963979 21158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.957985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:43403 (local address 127.12.234.1:46629)
0809 04:44:24.958327 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:24.958338 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:24.958365 (+ 27us) server_negotiation.cc:407] Connection header received
0809 04:44:24.958452 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:24.958458 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:24.958543 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:24.958657 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:24.959230 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.960220 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.962140 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.962943 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.963148 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:24.963235 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:24.963551 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 04:44:24.963684 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 04:44:24.964147 21171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.957701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:42793 (local address 127.12.234.2:53247)
0809 04:44:24.958592 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:24.958606 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:24.958760 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:24.959108 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:24.959115 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:24.959217 (+ 102us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:24.959490 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:24.959499 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.960668 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.960675 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:24.962079 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:24.962093 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.963237 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.963243 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:24.963413 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:24.963437 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 04:44:24.963545 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 04:44:24.963668 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":798,"mutex_wait_us":118,"spinlock_wait_cycles":17024,"thread_start_us":491,"threads_started":1}
I20260809 04:44:24.964459 21170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.957695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:46629 (local address 127.12.234.2:43403)
0809 04:44:24.958024 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:24.958057 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:24.958226 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:24.958699 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:24.958704 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:24.958757 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:24.959059 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:24.959073 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.960410 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.960419 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:24.961956 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:24.961971 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.963171 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.963179 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:24.963372 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:24.963399 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 04:44:24.963523 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 04:44:24.963675 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"spinlock_wait_cycles":7808}
I20260809 04:44:24.965194 20705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5dc2dd7b87b4abb8ac8da1cb390ee8f" candidate_uuid: "de0ed2dbaf5d438eae07171c965a3142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2833ba21f588473f8daa7d439c8d7518" is_pre_election: true
I20260809 04:44:24.965145 21009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5dc2dd7b87b4abb8ac8da1cb390ee8f" candidate_uuid: "de0ed2dbaf5d438eae07171c965a3142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "571f385b7e1143a5a32f5862960236c0" is_pre_election: true
I20260809 04:44:24.965483 20705 raft_consensus.cc:2454] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 2833ba21f588473f8daa7d439c8d7518 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de0ed2dbaf5d438eae07171c965a3142 in term 0.
I20260809 04:44:24.965615 21009 raft_consensus.cc:2454] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 571f385b7e1143a5a32f5862960236c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de0ed2dbaf5d438eae07171c965a3142 in term 0.
I20260809 04:44:24.966161 20791 leader_election.cc:302] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2833ba21f588473f8daa7d439c8d7518, de0ed2dbaf5d438eae07171c965a3142; no voters:
I20260809 04:44:24.966637 21169 raft_consensus.cc:2781] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:44:24.966781 21169 raft_consensus.cc:482] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:44:24.966884 21169 raft_consensus.cc:3037] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:24.970009 21169 raft_consensus.cc:504] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f385b7e1143a5a32f5862960236c0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42793 } } peers { permanent_uuid: "de0ed2dbaf5d438eae07171c965a3142" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 34979 } } peers { permanent_uuid: "2833ba21f588473f8daa7d439c8d7518" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 46629 } }
I20260809 04:44:24.970671 21169 leader_election.cc:288] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142 [CANDIDATE]: Term 1 election: Requested vote from peers 571f385b7e1143a5a32f5862960236c0 (127.12.234.3:42793), 2833ba21f588473f8daa7d439c8d7518 (127.12.234.1:46629)
I20260809 04:44:24.971068 21009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5dc2dd7b87b4abb8ac8da1cb390ee8f" candidate_uuid: "de0ed2dbaf5d438eae07171c965a3142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "571f385b7e1143a5a32f5862960236c0"
I20260809 04:44:24.971272 21009 raft_consensus.cc:3037] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 571f385b7e1143a5a32f5862960236c0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:24.971359 20705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5dc2dd7b87b4abb8ac8da1cb390ee8f" candidate_uuid: "de0ed2dbaf5d438eae07171c965a3142" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2833ba21f588473f8daa7d439c8d7518"
I20260809 04:44:24.971596 20705 raft_consensus.cc:3037] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 2833ba21f588473f8daa7d439c8d7518 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:24.973876 20705 raft_consensus.cc:2454] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 2833ba21f588473f8daa7d439c8d7518 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de0ed2dbaf5d438eae07171c965a3142 in term 1.
I20260809 04:44:24.973912 21009 raft_consensus.cc:2454] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 571f385b7e1143a5a32f5862960236c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de0ed2dbaf5d438eae07171c965a3142 in term 1.
I20260809 04:44:24.974327 20791 leader_election.cc:302] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2833ba21f588473f8daa7d439c8d7518, de0ed2dbaf5d438eae07171c965a3142; no voters:
I20260809 04:44:24.974601 21169 raft_consensus.cc:2781] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:24.975111 21169 raft_consensus.cc:686] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142 [term 1 LEADER]: Becoming Leader. State: Replica: de0ed2dbaf5d438eae07171c965a3142, State: Running, Role: LEADER
I20260809 04:44:24.975370 20911 heartbeater.cc:503] Master 127.12.234.61:41005 was elected leader, sending a full tablet report...
I20260809 04:44:24.975919 21169 consensus_queue.cc:237] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f385b7e1143a5a32f5862960236c0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42793 } } peers { permanent_uuid: "de0ed2dbaf5d438eae07171c965a3142" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 34979 } } peers { permanent_uuid: "2833ba21f588473f8daa7d439c8d7518" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 46629 } }
I20260809 04:44:24.976379 21170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.975839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.12.234.2:43917)
0809 04:44:24.976028 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:24.976177 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 04:44:24.979818 20356 catalog_manager.cc:5495] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142 reported cstate change: term changed from 0 to 1, leader changed from <none> to de0ed2dbaf5d438eae07171c965a3142 (127.12.234.2). New cstate: current_term: 1 leader_uuid: "de0ed2dbaf5d438eae07171c965a3142" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f385b7e1143a5a32f5862960236c0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de0ed2dbaf5d438eae07171c965a3142" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 34979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2833ba21f588473f8daa7d439c8d7518" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 46629 } health_report { overall_health: UNKNOWN } } }
I20260809 04:44:24.997042 21156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:24.991963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39218 (local address 127.12.234.2:34979)
0809 04:44:24.992214 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:24.992219 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:24.992235 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:44:24.992450 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:24.992453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:24.992530 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:24.992648 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:44:24.992955 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.993623 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.994873 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:24.995309 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:24.995420 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:24.995766 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:44:24.996411 (+ 645us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:44:24.996529 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:24.996616 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 04:44:24.996684 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 04:44:25.001168 20705 raft_consensus.cc:1261] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 2833ba21f588473f8daa7d439c8d7518 [term 1 FOLLOWER]: Refusing update from remote peer de0ed2dbaf5d438eae07171c965a3142: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:25.001294 21009 raft_consensus.cc:1261] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 571f385b7e1143a5a32f5862960236c0 [term 1 FOLLOWER]: Refusing update from remote peer de0ed2dbaf5d438eae07171c965a3142: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:25.002012 21172 consensus_queue.cc:1035] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2833ba21f588473f8daa7d439c8d7518" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 46629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:25.002401 21169 consensus_queue.cc:1035] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142 [LEADER]: Connected to new peer: Peer: permanent_uuid: "571f385b7e1143a5a32f5862960236c0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 42793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:25.005856 21167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:25.004614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.12.234.1:35473)
0809 04:44:25.005373 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:25.005518 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":602}
I20260809 04:44:25.006258 21163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:25.005570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.12.234.3:52973)
0809 04:44:25.005905 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:25.006038 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
I20260809 04:44:25.016223 21163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:25.015577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.12.234.3:58643)
0809 04:44:25.015859 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:25.015968 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260809 04:44:25.026110 21176 mvcc.cc:205] Tried to move back new op lower bound from 7316482723837530112 to 7316482723745783808. Current Snapshot: MvccSnapshot[applied={T|T < 7316482723837530112}]
I20260809 04:44:25.027611 21167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:25.027023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.12.234.1:39935)
0809 04:44:25.027310 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:25.027417 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
I20260809 04:44:25.037741 21170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:25.037105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.12.234.2:46039)
0809 04:44:25.037337 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:25.037433 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260809 04:44:25.039134 21170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:25.038607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.12.234.2:38449)
0809 04:44:25.038852 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:25.038945 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260809 04:44:25.048691 20356 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:25.048964 20356 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:25.055192 20356 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b4a48d1bc3cb43ef90c0916f666eec83: Sending DeleteTablet for 3 replicas of tablet d5dc2dd7b87b4abb8ac8da1cb390ee8f
I20260809 04:44:25.056190 20988 tablet_service.cc:1514] Processing DeleteTablet for tablet d5dc2dd7b87b4abb8ac8da1cb390ee8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:25 UTC) from {username='oryx', principal='oryx/127.12.234.61@KRBTEST.COM'} at 127.0.0.1:41138
I20260809 04:44:25.056767 20839 tablet_service.cc:1514] Processing DeleteTablet for tablet d5dc2dd7b87b4abb8ac8da1cb390ee8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:25 UTC) from {username='oryx', principal='oryx/127.12.234.61@KRBTEST.COM'} at 127.0.0.1:39208
I20260809 04:44:25.057036 21186 tablet_replica.cc:330] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 571f385b7e1143a5a32f5862960236c0: stopping tablet replica
I20260809 04:44:25.057228 21186 raft_consensus.cc:2229] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 571f385b7e1143a5a32f5862960236c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:44:25.057431 20684 tablet_service.cc:1514] Processing DeleteTablet for tablet d5dc2dd7b87b4abb8ac8da1cb390ee8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:25 UTC) from {username='oryx', principal='oryx/127.12.234.61@KRBTEST.COM'} at 127.0.0.1:54744
I20260809 04:44:25.057575 21186 raft_consensus.cc:2258] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 571f385b7e1143a5a32f5862960236c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:25.057699 21187 tablet_replica.cc:330] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142: stopping tablet replica
I20260809 04:44:25.057933 21187 raft_consensus.cc:2229] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:44:25.058599 21187 raft_consensus.cc:2258] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:25.058831 21186 ts_tablet_manager.cc:1905] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 571f385b7e1143a5a32f5862960236c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:25.060066 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 20529
I20260809 04:44:25.067492 21186 ts_tablet_manager.cc:1918] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 571f385b7e1143a5a32f5862960236c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:25.067670 21186 log.cc:1196] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 571f385b7e1143a5a32f5862960236c0: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-2/wal/wals/d5dc2dd7b87b4abb8ac8da1cb390ee8f
I20260809 04:44:25.068315 21186 ts_tablet_manager.cc:1939] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 571f385b7e1143a5a32f5862960236c0: Deleting consensus metadata
I20260809 04:44:25.069710 20342 catalog_manager.cc:4841] TS 571f385b7e1143a5a32f5862960236c0 (127.12.234.3:42793): tablet d5dc2dd7b87b4abb8ac8da1cb390ee8f (table test-table [id=9b869a5f3d8d4cd7ad069eec0c03750c]) successfully deleted
I20260809 04:44:25.072173 21188 tablet_replica.cc:330] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 2833ba21f588473f8daa7d439c8d7518: stopping tablet replica
I20260809 04:44:25.072369 21188 raft_consensus.cc:2229] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 2833ba21f588473f8daa7d439c8d7518 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:44:25.072708 21188 raft_consensus.cc:2258] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 2833ba21f588473f8daa7d439c8d7518 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:25.072717 21187 ts_tablet_manager.cc:1905] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:25.074059 21188 ts_tablet_manager.cc:1905] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 2833ba21f588473f8daa7d439c8d7518: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:25.077891 21187 ts_tablet_manager.cc:1918] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:25.078054 21187 log.cc:1196] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-1/wal/wals/d5dc2dd7b87b4abb8ac8da1cb390ee8f
I20260809 04:44:25.078584 21187 ts_tablet_manager.cc:1939] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P de0ed2dbaf5d438eae07171c965a3142: Deleting consensus metadata
I20260809 04:44:25.079722 20340 catalog_manager.cc:4841] TS de0ed2dbaf5d438eae07171c965a3142 (127.12.234.2:34979): tablet d5dc2dd7b87b4abb8ac8da1cb390ee8f (table test-table [id=9b869a5f3d8d4cd7ad069eec0c03750c]) successfully deleted
I20260809 04:44:25.080578 21188 ts_tablet_manager.cc:1918] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 2833ba21f588473f8daa7d439c8d7518: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:25.080714 21188 log.cc:1196] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 2833ba21f588473f8daa7d439c8d7518: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786250603326224-13224-0/minicluster-data/ts-0/wal/wals/d5dc2dd7b87b4abb8ac8da1cb390ee8f
I20260809 04:44:25.081277 21188 ts_tablet_manager.cc:1939] T d5dc2dd7b87b4abb8ac8da1cb390ee8f P 2833ba21f588473f8daa7d439c8d7518: Deleting consensus metadata
I20260809 04:44:25.082388 20341 catalog_manager.cc:4841] TS 2833ba21f588473f8daa7d439c8d7518 (127.12.234.1:46629): tablet d5dc2dd7b87b4abb8ac8da1cb390ee8f (table test-table [id=9b869a5f3d8d4cd7ad069eec0c03750c]) successfully deleted
I20260809 04:44:25.089241 21158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:25.060809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54746 (local address 127.12.234.1:46629)
0809 04:44:25.060994 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:25.061002 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:25.072595 (+ 11593us) server_negotiation.cc:407] Connection header received
0809 04:44:25.072999 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:25.073005 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:25.073105 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:25.073207 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:25.075172 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:25.076394 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:25.078114 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:25.078866 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:25.078994 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:25.082484 (+ 3490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:25.082503 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:25.082529 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:25.082577 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:25.086348 (+ 3771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:25.086982 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:25.086997 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:25.087001 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:25.087197 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:25.087544 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:25.087549 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:25.087552 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:25.088002 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:25.088377 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:25.088610 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 04:44:25.088864 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 04:44:25.089882 20749 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:25.200996 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 20529
I20260809 04:44:25.224656 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 20767
I20260809 04:44:25.240694 21156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:25.225474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39226 (local address 127.12.234.2:34979)
0809 04:44:25.225671 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:25.225677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:25.225700 (+ 23us) server_negotiation.cc:407] Connection header received
0809 04:44:25.225876 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:25.225886 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:25.226022 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:25.226137 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:25.227492 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:25.228764 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:25.230009 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:25.230678 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:25.230840 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:25.233512 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:25.233530 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:25.233554 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:25.233602 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:25.237645 (+ 4043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:25.238294 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:25.238305 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:25.238308 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:25.238504 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:25.238887 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:25.238893 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:25.238896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:25.239373 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:25.239697 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:25.239956 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 04:44:25.240207 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 04:44:25.241480 20904 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:25.275687 21138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:25.275208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.0.0.1:57616)
0809 04:44:25.275401 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:25.275519 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 04:44:25.276809 21138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:25.276398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44473 (local address 127.0.0.1:57622)
0809 04:44:25.276564 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:25.276653 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 04:44:25.363075 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 20767
I20260809 04:44:25.388206 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 20923
I20260809 04:44:25.403134 21154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:25.389022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41146 (local address 127.12.234.3:42793)
0809 04:44:25.389245 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:25.389254 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:25.389279 (+ 25us) server_negotiation.cc:407] Connection header received
0809 04:44:25.389515 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:25.389523 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:25.389610 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:25.389734 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:25.391175 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:25.391998 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:25.393389 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:25.393845 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:25.393995 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:25.396942 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:25.396965 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:25.396983 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:25.397024 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:25.399903 (+ 2879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:25.400916 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:25.400926 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:25.400930 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:25.401071 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:25.401468 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:25.401473 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:25.401476 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:25.401877 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:25.402207 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:25.402512 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 04:44:25.402779 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 04:44:25.403967 21052 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:25.530889 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 20923
I20260809 04:44:25.560850 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 20212
I20260809 04:44:25.561949 20388 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:25.688309 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 20212
I20260809 04:44:25.715489 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 20319
I20260809 04:44:25.716636 20593 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:25.835501 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 20319
2026-08-09T04:44:25Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9845 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:25 dist-test-slave-1jjb krb5kdc[21197](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:25 dist-test-slave-1jjb krb5kdc[21197](info): set up 2 sockets
Aug 09 04:44:25 dist-test-slave-1jjb krb5kdc[21197](info): commencing operation
krb5kdc: starting...
W20260809 04:44:26.948988 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.003s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:44:27 dist-test-slave-1jjb krb5kdc[21197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:27Z Disabled control of system clock
WARNING: no policy specified for oryx/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.62.keytab.
Entry for principal oryx/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.62.keytab.
I20260809 04:44:27.169306 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:33429
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.62.keytab
--principal=oryx/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:39211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:33429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:27.482625 21210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:27.482879 21210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:27.482962 21210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:27.489972 21210 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:27.490062 21210 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 04:44:27.490101 21210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:27.490160 21210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:27.490207 21210 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:27.490260 21210 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:27.499320 21210 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:39211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:33429
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.62.keytab
--principal=oryx/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:33429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:27.500612 21210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:27.501813 21210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:27.513494 21215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:27.513528 21219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:27.513528 21216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:27.514273 21210 server_base.cc:1034] running on GCE node
I20260809 04:44:27.514905 21210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:27.516002 21210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:27.517226 21210 hybrid_clock.cc:648] HybridClock initialized: now 1786250667517154 us; error 36 us; skew 500 ppm
I20260809 04:44:27.517597 21210 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:27 dist-test-slave-1jjb krb5kdc[21197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250667, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:27.520514 21210 init.cc:377] Logged in from keytab as oryx/127.12.234.62@KRBTEST.COM (short username oryx)
I20260809 04:44:27.521938 21210 webserver.cc:460] Webserver started at http://127.12.234.62:43891/ using document root <none> and password file <none>
I20260809 04:44:27.522609 21210 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:27.522706 21210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:27.522948 21210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:27.525570 21210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "c45f58836ad348a895b951ab8de06549"
format_stamp: "Formatted at 2026-08-09 04:44:27 on dist-test-slave-1jjb"
server_key: "923f94441d57926e5449b237d1a6807b"
server_key_iv: "e7c4f46efe81e7fea268fe7878898c54"
server_key_version: "encryptionkey@0"
I20260809 04:44:27.526219 21210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "c45f58836ad348a895b951ab8de06549"
format_stamp: "Formatted at 2026-08-09 04:44:27 on dist-test-slave-1jjb"
server_key: "923f94441d57926e5449b237d1a6807b"
server_key_iv: "e7c4f46efe81e7fea268fe7878898c54"
server_key_version: "encryptionkey@0"
I20260809 04:44:27.530959 21210 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 04:44:27.533972 21226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:27.535158 21210 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:44:27.535321 21210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "c45f58836ad348a895b951ab8de06549"
format_stamp: "Formatted at 2026-08-09 04:44:27 on dist-test-slave-1jjb"
server_key: "923f94441d57926e5449b237d1a6807b"
server_key_iv: "e7c4f46efe81e7fea268fe7878898c54"
server_key_version: "encryptionkey@0"
I20260809 04:44:27.535517 21210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:27.551939 21210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:27.555318 21210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:27.555584 21210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:27.577239 21210 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:33429
I20260809 04:44:27.577279 21292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:33429 every 8 connection(s)
I20260809 04:44:27.578606 21210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:27.582255 21293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:27.588194 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 21210
I20260809 04:44:27.588476 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:44:27.589423 13224 external_mini_cluster.cc:1428] Setting key b815be6e377db8447e63981dfb8caa51
I20260809 04:44:27.593534 21293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c45f58836ad348a895b951ab8de06549: Bootstrap starting.
I20260809 04:44:27.597225 21293 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c45f58836ad348a895b951ab8de06549: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:27.598546 21293 log.cc:824] T 00000000000000000000000000000000 P c45f58836ad348a895b951ab8de06549: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:27.601647 21293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c45f58836ad348a895b951ab8de06549: No bootstrap required, opened a new log
Aug 09 04:44:27 dist-test-slave-1jjb krb5kdc[21197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM
I20260809 04:44:27.612097 21293 raft_consensus.cc:348] T 00000000000000000000000000000000 P c45f58836ad348a895b951ab8de06549 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45f58836ad348a895b951ab8de06549" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 33429 } }
I20260809 04:44:27.612531 21293 raft_consensus.cc:374] T 00000000000000000000000000000000 P c45f58836ad348a895b951ab8de06549 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:27.612651 21293 raft_consensus.cc:729] T 00000000000000000000000000000000 P c45f58836ad348a895b951ab8de06549 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c45f58836ad348a895b951ab8de06549, State: Initialized, Role: FOLLOWER
I20260809 04:44:27.613345 21293 consensus_queue.cc:260] T 00000000000000000000000000000000 P c45f58836ad348a895b951ab8de06549 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45f58836ad348a895b951ab8de06549" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 33429 } }
I20260809 04:44:27.613642 21296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:27.591499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39208 (local address 127.12.234.62:33429)
0809 04:44:27.592269 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:27.592279 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:27.592347 (+ 68us) server_negotiation.cc:407] Connection header received
0809 04:44:27.593836 (+ 1489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:27.593856 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:27.594172 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:27.594430 (+ 258us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:27.596238 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:27.597586 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:27.599398 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:27.600206 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:27.600335 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:27.604885 (+ 4550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:27.604895 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:27.604907 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:27.604943 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:27.609071 (+ 4128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:27.609748 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:27.609759 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:27.609767 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:27.610014 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:27.610371 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:27.610377 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:27.610381 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:27.611108 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:27.611439 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:27.611760 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 04:44:27.612188 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":343,"threads_started":1}
I20260809 04:44:27.613667 21293 raft_consensus.cc:388] T 00000000000000000000000000000000 P c45f58836ad348a895b951ab8de06549 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:27.613821 21293 raft_consensus.cc:482] T 00000000000000000000000000000000 P c45f58836ad348a895b951ab8de06549 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:27.614025 21293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c45f58836ad348a895b951ab8de06549 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:27.616637 21293 raft_consensus.cc:504] T 00000000000000000000000000000000 P c45f58836ad348a895b951ab8de06549 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45f58836ad348a895b951ab8de06549" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 33429 } }
I20260809 04:44:27.617175 21293 leader_election.cc:302] T 00000000000000000000000000000000 P c45f58836ad348a895b951ab8de06549 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c45f58836ad348a895b951ab8de06549; no voters:
I20260809 04:44:27.617741 21293 leader_election.cc:288] T 00000000000000000000000000000000 P c45f58836ad348a895b951ab8de06549 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:27.617875 21298 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c45f58836ad348a895b951ab8de06549 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:27.618654 21298 raft_consensus.cc:686] T 00000000000000000000000000000000 P c45f58836ad348a895b951ab8de06549 [term 1 LEADER]: Becoming Leader. State: Replica: c45f58836ad348a895b951ab8de06549, State: Running, Role: LEADER
I20260809 04:44:27.619143 21298 consensus_queue.cc:237] T 00000000000000000000000000000000 P c45f58836ad348a895b951ab8de06549 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45f58836ad348a895b951ab8de06549" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 33429 } }
I20260809 04:44:27.619911 21293 sys_catalog.cc:564] T 00000000000000000000000000000000 P c45f58836ad348a895b951ab8de06549 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:27.621762 21300 sys_catalog.cc:455] T 00000000000000000000000000000000 P c45f58836ad348a895b951ab8de06549 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c45f58836ad348a895b951ab8de06549. Latest consensus state: current_term: 1 leader_uuid: "c45f58836ad348a895b951ab8de06549" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45f58836ad348a895b951ab8de06549" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 33429 } } }
I20260809 04:44:27.621869 21299 sys_catalog.cc:455] T 00000000000000000000000000000000 P c45f58836ad348a895b951ab8de06549 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c45f58836ad348a895b951ab8de06549" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c45f58836ad348a895b951ab8de06549" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 33429 } } }
I20260809 04:44:27.622200 21300 sys_catalog.cc:458] T 00000000000000000000000000000000 P c45f58836ad348a895b951ab8de06549 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:27.622273 21299 sys_catalog.cc:458] T 00000000000000000000000000000000 P c45f58836ad348a895b951ab8de06549 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:27.624586 21303 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:27.628998 21303 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:27.636783 21303 catalog_manager.cc:1347] Generated new cluster ID: c0e9db6e69224e0dae3556b42a83e3d1
I20260809 04:44:27.636893 21303 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:27.653894 21303 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:27.655081 21303 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:27.660341 21303 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c45f58836ad348a895b951ab8de06549: Generated new TSK 0
I20260809 04:44:27.661027 21303 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.12.234.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.234.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.234.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.1.keytab.
Entry for principal oryx/127.12.234.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.1.keytab.
I20260809 04:44:27.723064 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:0
--local_ip_for_outbound_sockets=127.12.234.1
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:33429
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.1.keytab
--principal=oryx/127.12.234.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:39211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:28.039199 21321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:28.039428 21321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:28.039597 21321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:28.046068 21321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:28.046190 21321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:28.046331 21321 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:44:28.046446 21321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:44:28.055071 21321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:39211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.1.keytab
--principal=oryx/127.12.234.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.234.62:33429
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:28.056159 21321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:28.057315 21321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:28.069873 21328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:28.069839 21330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:28.069929 21326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:28.070683 21321 server_base.cc:1034] running on GCE node
I20260809 04:44:28.071134 21321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:28.071807 21321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:28.073053 21321 hybrid_clock.cc:648] HybridClock initialized: now 1786250668072999 us; error 42 us; skew 500 ppm
I20260809 04:44:28.073326 21321 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:28 dist-test-slave-1jjb krb5kdc[21197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250668, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:28.076184 21321 init.cc:377] Logged in from keytab as oryx/127.12.234.1@KRBTEST.COM (short username oryx)
I20260809 04:44:28.077466 21321 webserver.cc:460] Webserver started at http://127.12.234.1:43045/ using document root <none> and password file <none>
I20260809 04:44:28.078121 21321 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:28.078251 21321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:28.078575 21321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:28.081204 21321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-0/data/instance:
uuid: "65056858c7d946919f793c452292c318"
format_stamp: "Formatted at 2026-08-09 04:44:28 on dist-test-slave-1jjb"
server_key: "528067ee338d15c330a999ddd8194492"
server_key_iv: "495fd10d6032b74887cebef3b0faccf1"
server_key_version: "encryptionkey@0"
I20260809 04:44:28.081897 21321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance:
uuid: "65056858c7d946919f793c452292c318"
format_stamp: "Formatted at 2026-08-09 04:44:28 on dist-test-slave-1jjb"
server_key: "528067ee338d15c330a999ddd8194492"
server_key_iv: "495fd10d6032b74887cebef3b0faccf1"
server_key_version: "encryptionkey@0"
I20260809 04:44:28.086568 21321 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:44:28.089874 21337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:28.091042 21321 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 04:44:28.091202 21321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "65056858c7d946919f793c452292c318"
format_stamp: "Formatted at 2026-08-09 04:44:28 on dist-test-slave-1jjb"
server_key: "528067ee338d15c330a999ddd8194492"
server_key_iv: "495fd10d6032b74887cebef3b0faccf1"
server_key_version: "encryptionkey@0"
I20260809 04:44:28.091395 21321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:28.110199 21321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:28.114784 21321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:28.115072 21321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:28.117568 21321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:28.117679 21321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:28.117846 21321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:28.117931 21321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:44:28 dist-test-slave-1jjb krb5kdc[21197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250668, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM
I20260809 04:44:28.143555 21346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:28.118197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:48319 (local address 127.12.234.62:33429)
0809 04:44:28.120060 (+ 1863us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:28.120065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:28.120100 (+ 35us) server_negotiation.cc:407] Connection header received
0809 04:44:28.122780 (+ 2680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:28.122787 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:28.122884 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:28.123030 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:28.125920 (+ 2890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:28.127043 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:28.130241 (+ 3198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:28.130772 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:28.130868 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:28.136141 (+ 5273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:28.136147 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:28.136156 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:28.136189 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:28.139784 (+ 3595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:28.140849 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:28.140856 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:28.140861 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:28.141009 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:28.141602 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:28.141606 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:28.141610 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:28.141972 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:28.142127 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:28.143017 (+ 890us) server_negotiation.cc:299] Negotiation successful
0809 04:44:28.143224 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1722,"thread_start_us":1610,"threads_started":1}
I20260809 04:44:28.144891 21348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:28.118740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:33429 (local address 127.12.234.1:48319)
0809 04:44:28.119780 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:28.119840 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:28.122462 (+ 2622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:28.123323 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:28.123334 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:28.124322 (+ 988us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:28.125566 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:28.125602 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:28.127374 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:28.127379 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:28.130068 (+ 2689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:28.130091 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:28.131037 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:28.131046 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:28.131385 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:28.132116 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:28.132166 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:28.135873 (+ 3707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:28.140022 (+ 4149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:28.140033 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:28.140049 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:28.140694 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:28.141193 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:28.141199 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:28.141205 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:28.141460 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:28.142102 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:28.142109 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:28.142601 (+ 492us) client_negotiation.cc:769] Sending connection context
0809 04:44:28.142926 (+ 325us) client_negotiation.cc:240] Negotiation successful
0809 04:44:28.143370 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":387,"threads_started":1}
I20260809 04:44:28.159799 21321 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:40729
I20260809 04:44:28.159895 21467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:40729 every 8 connection(s)
I20260809 04:44:28.161156 21321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:44:28.163440 21468 heartbeater.cc:347] Connected to a master server at 127.12.234.62:33429
I20260809 04:44:28.163654 21468 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:28.164340 21468 heartbeater.cc:511] Master 127.12.234.62:33429 requested a full tablet report, sending...
I20260809 04:44:28.166407 21247 ts_manager.cc:194] Registered new tserver with Master: 65056858c7d946919f793c452292c318 (127.12.234.1:40729)
I20260809 04:44:28.166592 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 21321
I20260809 04:44:28.166769 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance
I20260809 04:44:28.167549 13224 external_mini_cluster.cc:1428] Setting key 78aa4dc419a73fe91a83b3f7f2336eb8
I20260809 04:44:28.168134 21247 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.234.1@KRBTEST.COM'} at 127.12.234.1:48319
I20260809 04:44:28.168951 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:28.155875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:33429 (local address 127.12.234.1:34635)
0809 04:44:28.156715 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:28.156739 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:28.156905 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:28.157242 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:28.157248 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:28.157866 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:28.158198 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:28.158208 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:28.159226 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:28.159231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:28.160744 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:28.160757 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:28.161510 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:28.161515 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:28.161680 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:28.162308 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:28.162335 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:28.163255 (+ 920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:28.166177 (+ 2922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:28.166181 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:28.166184 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:28.166637 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:28.167041 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:28.167045 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:28.167049 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:28.167225 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:28.167899 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:28.167905 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:28.168106 (+ 201us) client_negotiation.cc:769] Sending connection context
0809 04:44:28.168267 (+ 161us) client_negotiation.cc:240] Negotiation successful
0809 04:44:28.168554 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":515,"threads_started":1}
I20260809 04:44:28.169788 21346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:28.155915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:34635 (local address 127.12.234.62:33429)
0809 04:44:28.156153 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:28.156159 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:28.156834 (+ 675us) server_negotiation.cc:407] Connection header received
0809 04:44:28.157013 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:28.157016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:28.157083 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:28.157208 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:28.158364 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:28.159105 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:28.160892 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:28.161345 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:28.161464 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:28.163442 (+ 1978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:28.163448 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:28.163453 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:28.163496 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:28.166054 (+ 2558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:28.166773 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:28.166780 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:28.166784 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:28.166913 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:28.167354 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:28.167360 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:28.167364 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:28.167759 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:28.167907 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:28.169397 (+ 1490us) server_negotiation.cc:299] Negotiation successful
0809 04:44:28.169582 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
WARNING: no policy specified for oryx/127.12.234.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.234.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.234.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.2.keytab.
Entry for principal oryx/127.12.234.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.2.keytab.
I20260809 04:44:28.220762 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:0
--local_ip_for_outbound_sockets=127.12.234.2
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:33429
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.2.keytab
--principal=oryx/127.12.234.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:39211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:28.526932 21473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:28.527177 21473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:28.527397 21473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:28.534322 21473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:28.534413 21473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:28.534565 21473 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:44:28.534680 21473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:44:28.544024 21473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:39211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.2.keytab
--principal=oryx/127.12.234.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.234.62:33429
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:28.545359 21473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:28.546671 21473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:28.559654 21478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:28.559684 21479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:28.559763 21481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:28.560483 21473 server_base.cc:1034] running on GCE node
I20260809 04:44:28.561132 21473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:28.562021 21473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:28.563256 21473 hybrid_clock.cc:648] HybridClock initialized: now 1786250668563189 us; error 43 us; skew 500 ppm
I20260809 04:44:28.563529 21473 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:28 dist-test-slave-1jjb krb5kdc[21197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250668, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:28.566452 21473 init.cc:377] Logged in from keytab as oryx/127.12.234.2@KRBTEST.COM (short username oryx)
I20260809 04:44:28.567950 21473 webserver.cc:460] Webserver started at http://127.12.234.2:45549/ using document root <none> and password file <none>
I20260809 04:44:28.568697 21473 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:28.568831 21473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:28.569185 21473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:28.571969 21473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-1/data/instance:
uuid: "76e2a575ab054736a27c05ecfe23b575"
format_stamp: "Formatted at 2026-08-09 04:44:28 on dist-test-slave-1jjb"
server_key: "3e7dd4775129fc432e7fcf421c411473"
server_key_iv: "eb35cf079c3115f3c6e0cbc376eb3ead"
server_key_version: "encryptionkey@0"
I20260809 04:44:28.572908 21473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance:
uuid: "76e2a575ab054736a27c05ecfe23b575"
format_stamp: "Formatted at 2026-08-09 04:44:28 on dist-test-slave-1jjb"
server_key: "3e7dd4775129fc432e7fcf421c411473"
server_key_iv: "eb35cf079c3115f3c6e0cbc376eb3ead"
server_key_version: "encryptionkey@0"
I20260809 04:44:28.577847 21473 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 04:44:28.581277 21488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:28.582803 21473 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:44:28.582959 21473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "76e2a575ab054736a27c05ecfe23b575"
format_stamp: "Formatted at 2026-08-09 04:44:28 on dist-test-slave-1jjb"
server_key: "3e7dd4775129fc432e7fcf421c411473"
server_key_iv: "eb35cf079c3115f3c6e0cbc376eb3ead"
server_key_version: "encryptionkey@0"
I20260809 04:44:28.583185 21473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:28.607772 21473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:28.613008 21473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:28.613281 21473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:28.615866 21473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:28.615967 21473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:28.616117 21473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:28.616200 21473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 09 04:44:28 dist-test-slave-1jjb krb5kdc[21197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250668, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM
I20260809 04:44:28.639981 21346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:28.616291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:60941 (local address 127.12.234.62:33429)
0809 04:44:28.616526 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:28.616535 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:28.617958 (+ 1423us) server_negotiation.cc:407] Connection header received
0809 04:44:28.620184 (+ 2226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:28.620192 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:28.620360 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:28.620470 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:28.622816 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:28.624209 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:28.625864 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:28.626761 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:28.626904 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:28.631357 (+ 4453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:28.631365 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:28.631372 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:28.631421 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:28.635529 (+ 4108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:28.636449 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:28.636457 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:28.636464 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:28.636671 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:28.637379 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:28.637385 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:28.637389 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:28.637730 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:28.637929 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:28.639337 (+ 1408us) server_negotiation.cc:299] Negotiation successful
0809 04:44:28.639591 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 04:44:28.640820 21497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:28.616923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:33429 (local address 127.12.234.2:60941)
0809 04:44:28.617777 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:28.617828 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:28.619820 (+ 1992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:28.620761 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:28.620772 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:28.621610 (+ 838us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:28.622464 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:28.622488 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:28.624393 (+ 1905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:28.624402 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:28.625711 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:28.625724 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:28.626987 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:28.626992 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:28.627167 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:28.627894 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:28.627918 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:28.631103 (+ 3185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:28.635685 (+ 4582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:28.635691 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:28.635700 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:28.636280 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:28.636848 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:28.636856 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:28.636862 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:28.637182 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:28.638053 (+ 871us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:28.638064 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:28.638576 (+ 512us) client_negotiation.cc:769] Sending connection context
0809 04:44:28.638884 (+ 308us) client_negotiation.cc:240] Negotiation successful
0809 04:44:28.639447 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":367,"threads_started":1}
I20260809 04:44:28.664640 21346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:28.646830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:50509 (local address 127.12.234.62:33429)
0809 04:44:28.647093 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:28.647100 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:28.647795 (+ 695us) server_negotiation.cc:407] Connection header received
0809 04:44:28.647965 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:28.647973 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:28.648114 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:28.648241 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:28.649880 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:28.651233 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:28.653334 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:28.654008 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:28.654186 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:28.657055 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:28.657064 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:28.657072 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:28.657116 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:28.661333 (+ 4217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:28.662340 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:28.662348 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:28.662353 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:28.662497 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:28.662884 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:28.662891 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:28.662896 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:28.663408 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:28.663579 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:28.663939 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 04:44:28.664239 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 04:44:28.664672 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:28.646890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:33429 (local address 127.12.234.2:50509)
0809 04:44:28.647660 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:28.647686 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:28.647834 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:28.648292 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:28.648311 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:28.649043 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:28.649687 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:28.649705 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:28.651396 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:28.651404 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:28.653080 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:28.653107 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:28.654242 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:28.654250 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:28.654585 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:28.655358 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:28.655387 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:28.656805 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:28.661513 (+ 4708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:28.661520 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:28.661525 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:28.662183 (+ 658us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:28.662635 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:28.662639 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:28.662641 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:28.662761 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:28.663531 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:28.663534 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:28.663686 (+ 152us) client_negotiation.cc:769] Sending connection context
0809 04:44:28.663800 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 04:44:28.664024 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":406,"threads_started":1}
I20260809 04:44:28.676551 21473 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:42223
I20260809 04:44:28.676597 21615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:42223 every 8 connection(s)
I20260809 04:44:28.677718 21473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:44:28.679610 21616 heartbeater.cc:347] Connected to a master server at 127.12.234.62:33429
I20260809 04:44:28.679848 21616 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:28.680501 21616 heartbeater.cc:511] Master 127.12.234.62:33429 requested a full tablet report, sending...
I20260809 04:44:28.681761 21246 ts_manager.cc:194] Registered new tserver with Master: 76e2a575ab054736a27c05ecfe23b575 (127.12.234.2:42223)
I20260809 04:44:28.682099 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 21473
I20260809 04:44:28.682274 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance
I20260809 04:44:28.682675 21246 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.234.2@KRBTEST.COM'} at 127.12.234.2:60941
I20260809 04:44:28.682726 13224 external_mini_cluster.cc:1428] Setting key 1457fe5d7b03d6690455e568366b3e59
WARNING: no policy specified for oryx/127.12.234.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.234.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.234.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.3.keytab.
Entry for principal oryx/127.12.234.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.3.keytab.
I20260809 04:44:28.734987 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:0
--local_ip_for_outbound_sockets=127.12.234.3
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:33429
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.3.keytab
--principal=oryx/127.12.234.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:39211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:29.039242 21620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:29.039458 21620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:29.039603 21620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:29.046028 21620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:29.046139 21620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:29.046275 21620 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 04:44:29.046389 21620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:44:29.054967 21620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:39211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.3.keytab
--principal=oryx/127.12.234.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.234.62:33429
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:29.056000 21620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:29.057214 21620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:29.069346 21628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:29.069324 21626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:29.069736 21625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:29.070066 21620 server_base.cc:1034] running on GCE node
I20260809 04:44:29.070717 21620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:29.071578 21620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:29.072813 21620 hybrid_clock.cc:648] HybridClock initialized: now 1786250669072744 us; error 40 us; skew 500 ppm
I20260809 04:44:29.073137 21620 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:29 dist-test-slave-1jjb krb5kdc[21197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250669, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:29.075873 21620 init.cc:377] Logged in from keytab as oryx/127.12.234.3@KRBTEST.COM (short username oryx)
I20260809 04:44:29.077121 21620 webserver.cc:460] Webserver started at http://127.12.234.3:42863/ using document root <none> and password file <none>
I20260809 04:44:29.077795 21620 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:29.077935 21620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:29.078265 21620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:29.080965 21620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-2/data/instance:
uuid: "b7ec484fb401414bb9ee34f2ec284b22"
format_stamp: "Formatted at 2026-08-09 04:44:29 on dist-test-slave-1jjb"
server_key: "4643b9878b785642f810c4756572b18a"
server_key_iv: "695b99e9ab03d76b673aa8a919551d44"
server_key_version: "encryptionkey@0"
I20260809 04:44:29.081684 21620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance:
uuid: "b7ec484fb401414bb9ee34f2ec284b22"
format_stamp: "Formatted at 2026-08-09 04:44:29 on dist-test-slave-1jjb"
server_key: "4643b9878b785642f810c4756572b18a"
server_key_iv: "695b99e9ab03d76b673aa8a919551d44"
server_key_version: "encryptionkey@0"
I20260809 04:44:29.086416 21620 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 04:44:29.089771 21636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:29.090909 21620 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:44:29.091080 21620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "b7ec484fb401414bb9ee34f2ec284b22"
format_stamp: "Formatted at 2026-08-09 04:44:29 on dist-test-slave-1jjb"
server_key: "4643b9878b785642f810c4756572b18a"
server_key_iv: "695b99e9ab03d76b673aa8a919551d44"
server_key_version: "encryptionkey@0"
I20260809 04:44:29.091307 21620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:29.111438 21620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:29.114965 21620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:29.115255 21620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:29.117790 21620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:29.117937 21620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:29.118115 21620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:29.118198 21620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:44:29 dist-test-slave-1jjb krb5kdc[21197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250669, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM
I20260809 04:44:29.139971 21346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:29.118593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:55907 (local address 127.12.234.62:33429)
0809 04:44:29.118803 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:29.118809 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:29.120269 (+ 1460us) server_negotiation.cc:407] Connection header received
0809 04:44:29.122301 (+ 2032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:29.122310 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:29.122428 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:29.122537 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:29.124661 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:29.125681 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:29.128110 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:29.128552 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:29.128675 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:29.133102 (+ 4427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:29.133110 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:29.133116 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:29.133154 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:29.136582 (+ 3428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:29.137479 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:29.137485 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:29.137491 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:29.137657 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:29.138292 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:29.138298 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:29.138304 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:29.138635 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:29.138801 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:29.139440 (+ 639us) server_negotiation.cc:299] Negotiation successful
0809 04:44:29.139664 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 04:44:29.140820 21645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:29.119151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:33429 (local address 127.12.234.3:55907)
0809 04:44:29.120092 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:29.120146 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:29.122023 (+ 1877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:29.122793 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:29.122800 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:29.123506 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:29.124407 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:29.124438 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:29.125823 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:29.125830 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:29.127931 (+ 2101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:29.127943 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:29.128804 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:29.128813 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:29.129005 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:29.129890 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:29.129916 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:29.132813 (+ 2897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:29.136738 (+ 3925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:29.136750 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:29.136768 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:29.137283 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:29.137797 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:29.137805 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:29.137812 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:29.138126 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:29.138760 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:29.138766 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:29.139061 (+ 295us) client_negotiation.cc:769] Sending connection context
0809 04:44:29.139338 (+ 277us) client_negotiation.cc:240] Negotiation successful
0809 04:44:29.139774 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":424,"threads_started":1}
I20260809 04:44:29.161166 21346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:29.145987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:40913 (local address 127.12.234.62:33429)
0809 04:44:29.146130 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:29.146150 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:29.146777 (+ 627us) server_negotiation.cc:407] Connection header received
0809 04:44:29.146968 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:29.146973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:29.147062 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:29.147177 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:29.148963 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:29.150244 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:29.152641 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:29.153271 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:29.153421 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:29.155010 (+ 1589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:29.155019 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:29.155026 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:29.155070 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:29.158416 (+ 3346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:29.159287 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:29.159292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:29.159295 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:29.159448 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:29.159794 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:29.159799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:29.159802 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:29.160042 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:29.160225 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:29.160611 (+ 386us) server_negotiation.cc:299] Negotiation successful
0809 04:44:29.160816 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 04:44:29.161240 21707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:29.146085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:33429 (local address 127.12.234.3:40913)
0809 04:44:29.146680 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:29.146694 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:29.146837 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:29.147268 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:29.147274 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:29.148187 (+ 913us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:29.148771 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:29.148788 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:29.150453 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:29.150464 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:29.152442 (+ 1978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:29.152462 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:29.153480 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:29.153484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:29.153583 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:29.154075 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:29.154090 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:29.154813 (+ 723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:29.158575 (+ 3762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:29.158579 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:29.158583 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:29.159158 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:29.159545 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:29.159549 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:29.159552 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:29.159684 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:29.160189 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:29.160194 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:29.160370 (+ 176us) client_negotiation.cc:769] Sending connection context
0809 04:44:29.160511 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 04:44:29.160821 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":354,"threads_started":1}
I20260809 04:44:29.171473 21468 heartbeater.cc:503] Master 127.12.234.62:33429 was elected leader, sending a full tablet report...
I20260809 04:44:29.175392 21620 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:33343
I20260809 04:44:29.175431 21764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:33343 every 8 connection(s)
I20260809 04:44:29.176538 21620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:44:29.178333 21765 heartbeater.cc:347] Connected to a master server at 127.12.234.62:33429
I20260809 04:44:29.178589 21765 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:29.179128 21765 heartbeater.cc:511] Master 127.12.234.62:33429 requested a full tablet report, sending...
I20260809 04:44:29.180284 21246 ts_manager.cc:194] Registered new tserver with Master: b7ec484fb401414bb9ee34f2ec284b22 (127.12.234.3:33343)
I20260809 04:44:29.181063 21246 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.234.3@KRBTEST.COM'} at 127.12.234.3:55907
I20260809 04:44:29.184975 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 21620
I20260809 04:44:29.185119 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance
I20260809 04:44:29.185559 13224 external_mini_cluster.cc:1428] Setting key 6c6993ada1527c68d23aee5f4f589ba0
I20260809 04:44:29.188287 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 04:44:29 dist-test-slave-1jjb krb5kdc[21197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250669, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 04:44:29 dist-test-slave-1jjb krb5kdc[21197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:29 dist-test-slave-1jjb krb5kdc[21197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:29.685359 21616 heartbeater.cc:503] Master 127.12.234.62:33429 was elected leader, sending a full tablet report...
I20260809 04:44:30.183141 21765 heartbeater.cc:503] Master 127.12.234.62:33429 was elected leader, sending a full tablet report...
W20260809 04:44:30.324491 21773 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.086s user 0.003s sys 0.014s
W20260809 04:44:30.325734 21346 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:29.232495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39214 (local address 127.12.234.62:33429)
0809 04:44:29.232720 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:29.232729 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:29.232970 (+ 241us) server_negotiation.cc:407] Connection header received
0809 04:44:29.233188 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:29.233192 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:29.233257 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:29.233361 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:29.234594 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:29.235352 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:29.236698 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:29.237056 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:29.237185 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:30.325401 (+1088216us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39214: BlockingRecv error: recv got EOF from 127.0.0.1:39214 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 09 04:44:30 dist-test-slave-1jjb krb5kdc[21197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250669, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM
I20260809 04:44:30.350582 21346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:30.329949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39224 (local address 127.12.234.62:33429)
0809 04:44:30.330178 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:30.330185 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:30.330554 (+ 369us) server_negotiation.cc:407] Connection header received
0809 04:44:30.330744 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:30.330747 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:30.330822 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:30.330903 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:30.332857 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.333624 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.335704 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.336160 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.336255 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:30.345180 (+ 8925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:30.345185 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:30.345188 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:30.345218 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:30.347907 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:30.348629 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:30.348635 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:30.348638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:30.348751 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:30.349166 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:30.349171 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:30.349176 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:30.349459 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:30.349767 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:30.350047 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 04:44:30.350275 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 04:44:30.354945 21246 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:44:30.368525 21246 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:44:30.418990 21784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:30.382025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:42223 (local address 127.0.0.1:44954)
0809 04:44:30.408608 (+ 26583us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:30.408632 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:30.408776 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:30.410290 (+ 1514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:30.410302 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:30.410355 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:30.410845 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:30.410863 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.413496 (+ 2633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.413504 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:30.416117 (+ 2613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:30.416130 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.418059 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.418065 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:30.418277 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:30.418332 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 04:44:30.418442 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 04:44:30.418644 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26393,"thread_start_us":26229,"threads_started":1}
I20260809 04:44:30.420300 21787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:30.382642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44954 (local address 127.12.234.2:42223)
0809 04:44:30.409462 (+ 26820us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:30.409475 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:30.409515 (+ 40us) server_negotiation.cc:407] Connection header received
0809 04:44:30.409637 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:30.409649 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:30.409962 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:30.411113 (+ 1151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:30.411369 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.413302 (+ 1933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.416299 (+ 2997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.417795 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.419459 (+ 1664us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:30.419688 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:30.419807 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 04:44:30.420000 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26629,"thread_start_us":26478,"threads_started":1}
I20260809 04:44:30.423206 21785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:30.381998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:40729 (local address 127.0.0.1:37078)
0809 04:44:30.400731 (+ 18733us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:30.400777 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:30.400964 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:30.412542 (+ 11578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:30.412549 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:30.412583 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:30.413072 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:30.413087 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.417062 (+ 3975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.417071 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:30.419479 (+ 2408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:30.419492 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.422043 (+ 2551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.422052 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:30.422305 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:30.422355 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 04:44:30.422476 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 04:44:30.422678 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18427,"mutex_wait_us":35,"thread_start_us":18261,"threads_started":1}
I20260809 04:44:30.424283 21542 tablet_service.cc:1467] Processing CreateTablet for tablet 3e90f7d6c65b444ea8b2e8e7d341cc0c (DEFAULT_TABLE table=test-table [id=40a95c5a5723416ba8328c034a72c77b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:30.425385 21788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:30.402534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:33343 (local address 127.0.0.1:57592)
0809 04:44:30.414026 (+ 11492us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:30.414045 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:30.414164 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:30.415557 (+ 1393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:30.415561 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:30.415584 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:30.415951 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:30.415961 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.417836 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.417844 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:30.422502 (+ 4658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:30.422518 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.424684 (+ 2166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.424690 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:30.424855 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:30.424885 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 04:44:30.424972 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 04:44:30.425093 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11343,"thread_start_us":14274,"threads_started":1}
I20260809 04:44:30.425827 21542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e90f7d6c65b444ea8b2e8e7d341cc0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:30.426159 21786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:30.382673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37078 (local address 127.12.234.1:40729)
0809 04:44:30.411684 (+ 29011us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:30.411695 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:30.411728 (+ 33us) server_negotiation.cc:407] Connection header received
0809 04:44:30.411849 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:30.411859 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:30.412305 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:30.413329 (+ 1024us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:30.413607 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.416693 (+ 3086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.419661 (+ 2968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.421723 (+ 2062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.425191 (+ 3468us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:30.425434 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:30.425509 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 04:44:30.425694 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28817,"thread_start_us":29417,"threads_started":1}
I20260809 04:44:30.426347 21789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:30.402674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57592 (local address 127.12.234.3:33343)
0809 04:44:30.414651 (+ 11977us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:30.414659 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:30.414685 (+ 26us) server_negotiation.cc:407] Connection header received
0809 04:44:30.414775 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:30.414781 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:30.415000 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:30.415184 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:30.416157 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.417661 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.422722 (+ 5061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.424189 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.424335 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:30.424558 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:30.425876 (+ 1318us) server_negotiation.cc:299] Negotiation successful
0809 04:44:30.426065 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11796,"thread_start_us":12655,"threads_started":1}
I20260809 04:44:30.429939 21688 tablet_service.cc:1467] Processing CreateTablet for tablet 3e90f7d6c65b444ea8b2e8e7d341cc0c (DEFAULT_TABLE table=test-table [id=40a95c5a5723416ba8328c034a72c77b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:30.430110 21392 tablet_service.cc:1467] Processing CreateTablet for tablet 3e90f7d6c65b444ea8b2e8e7d341cc0c (DEFAULT_TABLE table=test-table [id=40a95c5a5723416ba8328c034a72c77b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:30.431352 21688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e90f7d6c65b444ea8b2e8e7d341cc0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:30.431767 21392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e90f7d6c65b444ea8b2e8e7d341cc0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:30.451536 21791 tablet_bootstrap.cc:492] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 76e2a575ab054736a27c05ecfe23b575: Bootstrap starting.
I20260809 04:44:30.455864 21791 tablet_bootstrap.cc:654] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 76e2a575ab054736a27c05ecfe23b575: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:30.456768 21792 tablet_bootstrap.cc:492] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P b7ec484fb401414bb9ee34f2ec284b22: Bootstrap starting.
I20260809 04:44:30.457314 21791 log.cc:824] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 76e2a575ab054736a27c05ecfe23b575: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:30.461017 21793 tablet_bootstrap.cc:492] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318: Bootstrap starting.
I20260809 04:44:30.461233 21792 tablet_bootstrap.cc:654] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P b7ec484fb401414bb9ee34f2ec284b22: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:30.462596 21792 log.cc:824] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P b7ec484fb401414bb9ee34f2ec284b22: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:30.465337 21793 tablet_bootstrap.cc:654] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:30.466738 21793 log.cc:824] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:30.469102 21791 tablet_bootstrap.cc:492] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 76e2a575ab054736a27c05ecfe23b575: No bootstrap required, opened a new log
I20260809 04:44:30.469473 21791 ts_tablet_manager.cc:1397] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 76e2a575ab054736a27c05ecfe23b575: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.010s
I20260809 04:44:30.478638 21792 tablet_bootstrap.cc:492] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P b7ec484fb401414bb9ee34f2ec284b22: No bootstrap required, opened a new log
I20260809 04:44:30.478902 21793 tablet_bootstrap.cc:492] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318: No bootstrap required, opened a new log
I20260809 04:44:30.478904 21792 ts_tablet_manager.cc:1397] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P b7ec484fb401414bb9ee34f2ec284b22: Time spent bootstrapping tablet: real 0.022s user 0.007s sys 0.013s
I20260809 04:44:30.479226 21793 ts_tablet_manager.cc:1397] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.009s
I20260809 04:44:30.481316 21791 raft_consensus.cc:348] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:30.481815 21791 raft_consensus.cc:374] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:30.481952 21791 raft_consensus.cc:729] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76e2a575ab054736a27c05ecfe23b575, State: Initialized, Role: FOLLOWER
I20260809 04:44:30.483858 21791 consensus_queue.cc:260] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 76e2a575ab054736a27c05ecfe23b575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:30.491127 21792 raft_consensus.cc:348] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:30.491726 21792 raft_consensus.cc:374] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:30.491904 21792 raft_consensus.cc:729] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7ec484fb401414bb9ee34f2ec284b22, State: Initialized, Role: FOLLOWER
I20260809 04:44:30.492048 21791 ts_tablet_manager.cc:1428] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 76e2a575ab054736a27c05ecfe23b575: Time spent starting tablet: real 0.022s user 0.016s sys 0.003s
I20260809 04:44:30.492342 21793 raft_consensus.cc:348] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:30.492736 21792 consensus_queue.cc:260] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P b7ec484fb401414bb9ee34f2ec284b22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:30.493003 21793 raft_consensus.cc:374] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:30.493165 21793 raft_consensus.cc:729] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65056858c7d946919f793c452292c318, State: Initialized, Role: FOLLOWER
I20260809 04:44:30.493960 21793 consensus_queue.cc:260] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:30.500921 21792 ts_tablet_manager.cc:1428] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P b7ec484fb401414bb9ee34f2ec284b22: Time spent starting tablet: real 0.022s user 0.016s sys 0.006s
I20260809 04:44:30.501206 21793 ts_tablet_manager.cc:1428] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318: Time spent starting tablet: real 0.022s user 0.020s sys 0.000s
W20260809 04:44:30.664940 21469 tablet.cc:2369] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:44:30.679008 21766 tablet.cc:2369] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P b7ec484fb401414bb9ee34f2ec284b22: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:44:30.680459 21617 tablet.cc:2369] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 76e2a575ab054736a27c05ecfe23b575: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:44:30.783923 21800 raft_consensus.cc:482] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:44:30.784287 21800 raft_consensus.cc:504] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:30.786211 21800 leader_election.cc:288] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76e2a575ab054736a27c05ecfe23b575 (127.12.234.2:42223), b7ec484fb401414bb9ee34f2ec284b22 (127.12.234.3:33343)
I20260809 04:44:30.792814 21789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:30.786755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:41655 (local address 127.12.234.3:33343)
0809 04:44:30.786996 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:30.787004 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:30.787423 (+ 419us) server_negotiation.cc:407] Connection header received
0809 04:44:30.787652 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:30.787658 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:30.787747 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:30.787848 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:30.788542 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.789296 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.791146 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.791885 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.792073 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:30.792167 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:30.792446 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 04:44:30.792578 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 04:44:30.792881 21801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:30.786472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:33343 (local address 127.12.234.1:41655)
0809 04:44:30.787284 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:30.787307 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:30.787494 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:30.787866 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:30.787870 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:30.787895 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:30.788386 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:30.788396 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.789433 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.789439 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:30.790984 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:30.790996 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.792121 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.792128 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:30.792282 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:30.792304 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 04:44:30.792421 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 04:44:30.792543 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":355,"threads_started":1}
I20260809 04:44:30.793217 21787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:30.786450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:50191 (local address 127.12.234.2:42223)
0809 04:44:30.786653 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:30.786661 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:30.787451 (+ 790us) server_negotiation.cc:407] Connection header received
0809 04:44:30.787679 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:30.787684 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:30.787762 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:30.787883 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:30.788666 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.789432 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.791314 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.792179 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.792359 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:30.792451 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:30.792776 (+ 325us) server_negotiation.cc:299] Negotiation successful
0809 04:44:30.792917 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:44:30.793279 21802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:30.786567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:42223 (local address 127.12.234.1:50191)
0809 04:44:30.787307 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:30.787328 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:30.787521 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:30.787897 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:30.787902 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:30.787928 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:30.788487 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:30.788498 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.789592 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.789599 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:30.791133 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:30.791147 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.792455 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.792464 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:30.792603 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:30.792625 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 04:44:30.792748 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 04:44:30.792875 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":475,"threads_started":1}
I20260809 04:44:30.794054 21718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e90f7d6c65b444ea8b2e8e7d341cc0c" candidate_uuid: "65056858c7d946919f793c452292c318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ec484fb401414bb9ee34f2ec284b22" is_pre_election: true
I20260809 04:44:30.794687 21718 raft_consensus.cc:2454] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65056858c7d946919f793c452292c318 in term 0.
I20260809 04:44:30.794695 21569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e90f7d6c65b444ea8b2e8e7d341cc0c" candidate_uuid: "65056858c7d946919f793c452292c318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76e2a575ab054736a27c05ecfe23b575" is_pre_election: true
I20260809 04:44:30.795272 21569 raft_consensus.cc:2454] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65056858c7d946919f793c452292c318 in term 0.
I20260809 04:44:30.795349 21340 leader_election.cc:302] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65056858c7d946919f793c452292c318, b7ec484fb401414bb9ee34f2ec284b22; no voters:
I20260809 04:44:30.795770 21800 raft_consensus.cc:2781] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:44:30.795970 21800 raft_consensus.cc:482] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:44:30.796151 21800 raft_consensus.cc:3037] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:30.798547 21800 raft_consensus.cc:504] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:30.799291 21800 leader_election.cc:288] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318 [CANDIDATE]: Term 1 election: Requested vote from peers 76e2a575ab054736a27c05ecfe23b575 (127.12.234.2:42223), b7ec484fb401414bb9ee34f2ec284b22 (127.12.234.3:33343)
I20260809 04:44:30.799644 21569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e90f7d6c65b444ea8b2e8e7d341cc0c" candidate_uuid: "65056858c7d946919f793c452292c318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76e2a575ab054736a27c05ecfe23b575"
I20260809 04:44:30.799774 21718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e90f7d6c65b444ea8b2e8e7d341cc0c" candidate_uuid: "65056858c7d946919f793c452292c318" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ec484fb401414bb9ee34f2ec284b22"
I20260809 04:44:30.799872 21569 raft_consensus.cc:3037] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:30.800024 21718 raft_consensus.cc:3037] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:30.802220 21718 raft_consensus.cc:2454] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P b7ec484fb401414bb9ee34f2ec284b22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65056858c7d946919f793c452292c318 in term 1.
I20260809 04:44:30.802237 21569 raft_consensus.cc:2454] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 76e2a575ab054736a27c05ecfe23b575 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65056858c7d946919f793c452292c318 in term 1.
I20260809 04:44:30.802625 21340 leader_election.cc:302] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65056858c7d946919f793c452292c318, b7ec484fb401414bb9ee34f2ec284b22; no voters:
I20260809 04:44:30.802929 21800 raft_consensus.cc:2781] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:30.803359 21800 raft_consensus.cc:686] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318 [term 1 LEADER]: Becoming Leader. State: Replica: 65056858c7d946919f793c452292c318, State: Running, Role: LEADER
I20260809 04:44:30.803931 21800 consensus_queue.cc:237] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:30.808893 21246 catalog_manager.cc:5495] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318 reported cstate change: term changed from 0 to 1, leader changed from <none> to 65056858c7d946919f793c452292c318 (127.12.234.1). New cstate: current_term: 1 leader_uuid: "65056858c7d946919f793c452292c318" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } health_report { overall_health: UNKNOWN } } }
I20260809 04:44:30.847159 21786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:30.842467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37094 (local address 127.12.234.1:40729)
0809 04:44:30.842645 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:30.842649 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:30.842683 (+ 34us) server_negotiation.cc:407] Connection header received
0809 04:44:30.842785 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:30.842788 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:30.842885 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:30.843053 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:44:30.843423 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.844207 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.845386 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:30.845706 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:30.845784 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:30.846137 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:44:30.846600 (+ 463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:44:30.846690 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:30.846797 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 04:44:30.846853 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 04:44:30.855015 21569 raft_consensus.cc:1261] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 76e2a575ab054736a27c05ecfe23b575 [term 1 FOLLOWER]: Refusing update from remote peer 65056858c7d946919f793c452292c318: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:30.855007 21718 raft_consensus.cc:1261] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P b7ec484fb401414bb9ee34f2ec284b22 [term 1 FOLLOWER]: Refusing update from remote peer 65056858c7d946919f793c452292c318: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:30.855949 21803 consensus_queue.cc:1035] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:30.856395 21800 consensus_queue.cc:1035] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:30.866685 21806 mvcc.cc:205] Tried to move back new op lower bound from 7316482747813171200 to 7316482747617095680. Current Snapshot: MvccSnapshot[applied={T|T < 7316482747813171200}]
I20260809 04:44:30.878901 21247 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:30.879148 21247 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:30.882350 21247 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c45f58836ad348a895b951ab8de06549: Sending DeleteTablet for 3 replicas of tablet 3e90f7d6c65b444ea8b2e8e7d341cc0c
I20260809 04:44:30.883332 21542 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e90f7d6c65b444ea8b2e8e7d341cc0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:30 UTC) from {username='oryx', principal='oryx/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:44954
I20260809 04:44:30.883569 21688 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e90f7d6c65b444ea8b2e8e7d341cc0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:30 UTC) from {username='oryx', principal='oryx/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:57592
I20260809 04:44:30.883728 21392 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e90f7d6c65b444ea8b2e8e7d341cc0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:30 UTC) from {username='oryx', principal='oryx/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:37078
I20260809 04:44:30.886605 21247 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:44:30.888636 21247 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:44:30.898586 21391 tablet_service.cc:1467] Processing CreateTablet for tablet 3ea9911969704119b3b7ab2972c07682 (DEFAULT_TABLE table=test-table [id=4f0f8324669e4e0490c7a0f0bfdc7df3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:30.899191 21391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ea9911969704119b3b7ab2972c07682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:30.899778 21818 tablet_replica.cc:330] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318: stopping tablet replica
I20260809 04:44:30.900274 21818 raft_consensus.cc:2229] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:44:30.901196 21818 raft_consensus.cc:2258] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:30.903643 21818 ts_tablet_manager.cc:1905] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:30.904580 21540 tablet_service.cc:1467] Processing CreateTablet for tablet 3ea9911969704119b3b7ab2972c07682 (DEFAULT_TABLE table=test-table [id=4f0f8324669e4e0490c7a0f0bfdc7df3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:30.905098 21540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ea9911969704119b3b7ab2972c07682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:30.909175 21687 tablet_service.cc:1467] Processing CreateTablet for tablet 3ea9911969704119b3b7ab2972c07682 (DEFAULT_TABLE table=test-table [id=4f0f8324669e4e0490c7a0f0bfdc7df3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:30.909768 21687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3ea9911969704119b3b7ab2972c07682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:30.909832 21816 tablet_replica.cc:330] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 76e2a575ab054736a27c05ecfe23b575: stopping tablet replica
I20260809 04:44:30.910223 21816 raft_consensus.cc:2229] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 76e2a575ab054736a27c05ecfe23b575 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:44:30.911628 21818 ts_tablet_manager.cc:1918] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:30.911660 21816 raft_consensus.cc:2258] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 76e2a575ab054736a27c05ecfe23b575 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:30.911783 21818 log.cc:1196] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-0/wal/wals/3e90f7d6c65b444ea8b2e8e7d341cc0c
I20260809 04:44:30.912842 21818 ts_tablet_manager.cc:1939] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 65056858c7d946919f793c452292c318: Deleting consensus metadata
I20260809 04:44:30.913642 21816 ts_tablet_manager.cc:1905] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 76e2a575ab054736a27c05ecfe23b575: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:30.914934 21230 catalog_manager.cc:4841] TS 65056858c7d946919f793c452292c318 (127.12.234.1:40729): tablet 3e90f7d6c65b444ea8b2e8e7d341cc0c (table test-table [id=40a95c5a5723416ba8328c034a72c77b]) successfully deleted
I20260809 04:44:30.917228 21791 tablet_bootstrap.cc:492] T 3ea9911969704119b3b7ab2972c07682 P 76e2a575ab054736a27c05ecfe23b575: Bootstrap starting.
I20260809 04:44:30.918324 21792 tablet_bootstrap.cc:492] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22: Bootstrap starting.
I20260809 04:44:30.918879 21791 tablet_bootstrap.cc:654] T 3ea9911969704119b3b7ab2972c07682 P 76e2a575ab054736a27c05ecfe23b575: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:30.919804 21792 tablet_bootstrap.cc:654] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:30.921334 21793 tablet_bootstrap.cc:492] T 3ea9911969704119b3b7ab2972c07682 P 65056858c7d946919f793c452292c318: Bootstrap starting.
I20260809 04:44:30.921938 21817 tablet_replica.cc:330] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P b7ec484fb401414bb9ee34f2ec284b22: stopping tablet replica
I20260809 04:44:30.922376 21817 raft_consensus.cc:2229] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P b7ec484fb401414bb9ee34f2ec284b22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:44:30.922804 21817 raft_consensus.cc:2258] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P b7ec484fb401414bb9ee34f2ec284b22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:30.923094 21793 tablet_bootstrap.cc:654] T 3ea9911969704119b3b7ab2972c07682 P 65056858c7d946919f793c452292c318: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:30.926522 21817 ts_tablet_manager.cc:1905] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P b7ec484fb401414bb9ee34f2ec284b22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:30.934656 21791 tablet_bootstrap.cc:492] T 3ea9911969704119b3b7ab2972c07682 P 76e2a575ab054736a27c05ecfe23b575: No bootstrap required, opened a new log
I20260809 04:44:30.934850 21791 ts_tablet_manager.cc:1397] T 3ea9911969704119b3b7ab2972c07682 P 76e2a575ab054736a27c05ecfe23b575: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.009s
I20260809 04:44:30.935218 21816 ts_tablet_manager.cc:1918] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 76e2a575ab054736a27c05ecfe23b575: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:30.935403 21816 log.cc:1196] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 76e2a575ab054736a27c05ecfe23b575: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-1/wal/wals/3e90f7d6c65b444ea8b2e8e7d341cc0c
I20260809 04:44:30.935950 21791 raft_consensus.cc:348] T 3ea9911969704119b3b7ab2972c07682 P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:30.936247 21791 raft_consensus.cc:374] T 3ea9911969704119b3b7ab2972c07682 P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:30.936388 21791 raft_consensus.cc:729] T 3ea9911969704119b3b7ab2972c07682 P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76e2a575ab054736a27c05ecfe23b575, State: Initialized, Role: FOLLOWER
I20260809 04:44:30.936522 21792 tablet_bootstrap.cc:492] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22: No bootstrap required, opened a new log
I20260809 04:44:30.936527 21793 tablet_bootstrap.cc:492] T 3ea9911969704119b3b7ab2972c07682 P 65056858c7d946919f793c452292c318: No bootstrap required, opened a new log
I20260809 04:44:30.936663 21792 ts_tablet_manager.cc:1397] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.000s
I20260809 04:44:30.936668 21793 ts_tablet_manager.cc:1397] T 3ea9911969704119b3b7ab2972c07682 P 65056858c7d946919f793c452292c318: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.007s
I20260809 04:44:30.936792 21791 consensus_queue.cc:260] T 3ea9911969704119b3b7ab2972c07682 P 76e2a575ab054736a27c05ecfe23b575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:30.937379 21791 ts_tablet_manager.cc:1428] T 3ea9911969704119b3b7ab2972c07682 P 76e2a575ab054736a27c05ecfe23b575: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 04:44:30.937462 21792 raft_consensus.cc:348] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:30.937693 21792 raft_consensus.cc:374] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:30.937736 21816 ts_tablet_manager.cc:1939] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P 76e2a575ab054736a27c05ecfe23b575: Deleting consensus metadata
I20260809 04:44:30.937803 21792 raft_consensus.cc:729] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7ec484fb401414bb9ee34f2ec284b22, State: Initialized, Role: FOLLOWER
I20260809 04:44:30.938067 21792 consensus_queue.cc:260] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:30.938690 21792 ts_tablet_manager.cc:1428] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 04:44:30.939661 21793 raft_consensus.cc:348] T 3ea9911969704119b3b7ab2972c07682 P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:30.939872 21793 raft_consensus.cc:374] T 3ea9911969704119b3b7ab2972c07682 P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:30.939985 21793 raft_consensus.cc:729] T 3ea9911969704119b3b7ab2972c07682 P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65056858c7d946919f793c452292c318, State: Initialized, Role: FOLLOWER
I20260809 04:44:30.940339 21793 consensus_queue.cc:260] T 3ea9911969704119b3b7ab2972c07682 P 65056858c7d946919f793c452292c318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:30.941486 21793 ts_tablet_manager.cc:1428] T 3ea9911969704119b3b7ab2972c07682 P 65056858c7d946919f793c452292c318: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260809 04:44:30.943810 21817 ts_tablet_manager.cc:1918] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P b7ec484fb401414bb9ee34f2ec284b22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:30.944015 21817 log.cc:1196] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P b7ec484fb401414bb9ee34f2ec284b22: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-2/wal/wals/3e90f7d6c65b444ea8b2e8e7d341cc0c
I20260809 04:44:30.944772 21817 ts_tablet_manager.cc:1939] T 3e90f7d6c65b444ea8b2e8e7d341cc0c P b7ec484fb401414bb9ee34f2ec284b22: Deleting consensus metadata
I20260809 04:44:30.944883 21229 catalog_manager.cc:4841] TS 76e2a575ab054736a27c05ecfe23b575 (127.12.234.2:42223): tablet 3e90f7d6c65b444ea8b2e8e7d341cc0c (table test-table [id=40a95c5a5723416ba8328c034a72c77b]) successfully deleted
I20260809 04:44:30.946328 21230 catalog_manager.cc:4841] TS b7ec484fb401414bb9ee34f2ec284b22 (127.12.234.3:33343): tablet 3e90f7d6c65b444ea8b2e8e7d341cc0c (table test-table [id=40a95c5a5723416ba8328c034a72c77b]) successfully deleted
I20260809 04:44:31.072961 21799 raft_consensus.cc:482] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:44:31.073288 21799 raft_consensus.cc:504] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:31.076027 21799 leader_election.cc:288] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 65056858c7d946919f793c452292c318 (127.12.234.1:40729), 76e2a575ab054736a27c05ecfe23b575 (127.12.234.2:42223)
I20260809 04:44:31.085366 21786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:31.076549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:57943 (local address 127.12.234.1:40729)
0809 04:44:31.076733 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:31.076740 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:31.078024 (+ 1284us) server_negotiation.cc:407] Connection header received
0809 04:44:31.078295 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:31.078301 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:31.078387 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:31.078493 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:31.079384 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.080903 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.082745 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.084344 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.084570 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:31.084657 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:31.084898 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 04:44:31.085034 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 04:44:31.085414 21823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:31.076165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:40729 (local address 127.12.234.3:57943)
0809 04:44:31.077889 (+ 1724us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:31.077906 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:31.078123 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:31.078700 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:31.078705 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:31.078728 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:31.079208 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:31.079217 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.081030 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.081035 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:31.082559 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:31.082570 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.084579 (+ 2009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.084586 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:31.084723 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:31.084753 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 04:44:31.084859 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 04:44:31.084979 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1584,"mutex_wait_us":59,"thread_start_us":1067,"threads_started":1}
I20260809 04:44:31.085800 21787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:31.076801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:41605 (local address 127.12.234.2:42223)
0809 04:44:31.076987 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:31.076991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:31.078343 (+ 1352us) server_negotiation.cc:407] Connection header received
0809 04:44:31.078522 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:31.078527 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:31.078623 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:31.078709 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:31.079618 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.080798 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.083178 (+ 2380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.084518 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.084683 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:31.084753 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:31.085375 (+ 622us) server_negotiation.cc:299] Negotiation successful
0809 04:44:31.085507 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:44:31.086467 21822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:31.076396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:42223 (local address 127.12.234.3:41605)
0809 04:44:31.078196 (+ 1800us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:31.078225 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:31.078377 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:31.078774 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:31.078778 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:31.078828 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:31.079385 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:31.079408 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.080926 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.080930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:31.083006 (+ 2076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:31.083020 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.084963 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.084970 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:31.085179 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:31.085212 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 04:44:31.085985 (+ 773us) client_negotiation.cc:240] Negotiation successful
0809 04:44:31.086110 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1516,"mutex_wait_us":45,"thread_start_us":1074,"threads_started":1}
I20260809 04:44:31.086584 21413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ea9911969704119b3b7ab2972c07682" candidate_uuid: "b7ec484fb401414bb9ee34f2ec284b22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65056858c7d946919f793c452292c318" is_pre_election: true
I20260809 04:44:31.087025 21413 raft_consensus.cc:2454] T 3ea9911969704119b3b7ab2972c07682 P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7ec484fb401414bb9ee34f2ec284b22 in term 0.
I20260809 04:44:31.087169 21569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ea9911969704119b3b7ab2972c07682" candidate_uuid: "b7ec484fb401414bb9ee34f2ec284b22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76e2a575ab054736a27c05ecfe23b575" is_pre_election: true
I20260809 04:44:31.087430 21569 raft_consensus.cc:2454] T 3ea9911969704119b3b7ab2972c07682 P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7ec484fb401414bb9ee34f2ec284b22 in term 0.
I20260809 04:44:31.087694 21639 leader_election.cc:302] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65056858c7d946919f793c452292c318, b7ec484fb401414bb9ee34f2ec284b22; no voters:
I20260809 04:44:31.088126 21799 raft_consensus.cc:2781] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:44:31.088264 21799 raft_consensus.cc:482] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:44:31.088411 21799 raft_consensus.cc:3037] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:31.091524 21799 raft_consensus.cc:504] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:31.092201 21799 leader_election.cc:288] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22 [CANDIDATE]: Term 1 election: Requested vote from peers 65056858c7d946919f793c452292c318 (127.12.234.1:40729), 76e2a575ab054736a27c05ecfe23b575 (127.12.234.2:42223)
I20260809 04:44:31.092648 21569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ea9911969704119b3b7ab2972c07682" candidate_uuid: "b7ec484fb401414bb9ee34f2ec284b22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76e2a575ab054736a27c05ecfe23b575"
I20260809 04:44:31.092644 21413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3ea9911969704119b3b7ab2972c07682" candidate_uuid: "b7ec484fb401414bb9ee34f2ec284b22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65056858c7d946919f793c452292c318"
I20260809 04:44:31.092829 21569 raft_consensus.cc:3037] T 3ea9911969704119b3b7ab2972c07682 P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:31.092842 21413 raft_consensus.cc:3037] T 3ea9911969704119b3b7ab2972c07682 P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:31.095064 21569 raft_consensus.cc:2454] T 3ea9911969704119b3b7ab2972c07682 P 76e2a575ab054736a27c05ecfe23b575 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7ec484fb401414bb9ee34f2ec284b22 in term 1.
I20260809 04:44:31.095966 21638 leader_election.cc:302] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76e2a575ab054736a27c05ecfe23b575, b7ec484fb401414bb9ee34f2ec284b22; no voters:
I20260809 04:44:31.096050 21413 raft_consensus.cc:2454] T 3ea9911969704119b3b7ab2972c07682 P 65056858c7d946919f793c452292c318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7ec484fb401414bb9ee34f2ec284b22 in term 1.
I20260809 04:44:31.096427 21799 raft_consensus.cc:2781] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:31.096978 21799 raft_consensus.cc:686] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22 [term 1 LEADER]: Becoming Leader. State: Replica: b7ec484fb401414bb9ee34f2ec284b22, State: Running, Role: LEADER
I20260809 04:44:31.097579 21799 consensus_queue.cc:237] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:31.100788 21247 catalog_manager.cc:5495] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7ec484fb401414bb9ee34f2ec284b22 (127.12.234.3). New cstate: current_term: 1 leader_uuid: "b7ec484fb401414bb9ee34f2ec284b22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } health_report { overall_health: HEALTHY } } }
Aug 09 04:44:31 dist-test-slave-1jjb krb5kdc[21197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250667, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.62@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM
I20260809 04:44:31.181128 21831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:31.166271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39226 (local address 127.12.234.62:33429)
0809 04:44:31.166918 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:31.166926 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:31.167481 (+ 555us) server_negotiation.cc:407] Connection header received
0809 04:44:31.167681 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:31.167686 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:31.167751 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:31.167864 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:31.169001 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.169799 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.171022 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.171556 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.171688 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:31.174993 (+ 3305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:31.174999 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:31.175003 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:31.175036 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:31.178349 (+ 3313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:31.179157 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:31.179163 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:31.179166 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:31.179283 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:31.179672 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:31.179679 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:31.179682 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:31.179995 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:31.180125 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:31.180607 (+ 482us) server_negotiation.cc:299] Negotiation successful
0809 04:44:31.180796 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":320,"threads_started":1}
I20260809 04:44:31.181891 21832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:31.166296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:33429 (local address 127.0.0.1:39226)
0809 04:44:31.167337 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:31.167361 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:31.167562 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:31.167887 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:31.167891 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:31.168639 (+ 748us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:31.168847 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:31.168856 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.169950 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.169956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:31.170887 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:31.170898 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.171724 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.171729 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:31.171928 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:31.172659 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:31.172676 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:31.174799 (+ 2123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:31.178511 (+ 3712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:31.178522 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:31.178530 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:31.179041 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:31.179414 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:31.179417 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:31.179420 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:31.179563 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:31.180207 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:31.180214 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:31.180371 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 04:44:31.181304 (+ 933us) client_negotiation.cc:240] Negotiation successful
0809 04:44:31.181530 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":926,"thread_start_us":828,"threads_started":1}
I20260809 04:44:31.187090 21247 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:39226:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 04:44:31.188344 21247 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:44:31.195902 21391 tablet_service.cc:1467] Processing CreateTablet for tablet 29cd2c628097436db297902e5fa7c6a1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=88130a2605064127a75b332c843f1645]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:44:31.196103 21540 tablet_service.cc:1467] Processing CreateTablet for tablet 29cd2c628097436db297902e5fa7c6a1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=88130a2605064127a75b332c843f1645]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:44:31.196507 21391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29cd2c628097436db297902e5fa7c6a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:31.196693 21540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29cd2c628097436db297902e5fa7c6a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:31.201025 21688 tablet_service.cc:1467] Processing CreateTablet for tablet 29cd2c628097436db297902e5fa7c6a1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=88130a2605064127a75b332c843f1645]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 04:44:31.201531 21688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29cd2c628097436db297902e5fa7c6a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:31.203501 21791 tablet_bootstrap.cc:492] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575: Bootstrap starting.
I20260809 04:44:31.205165 21791 tablet_bootstrap.cc:654] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:31.210824 21792 tablet_bootstrap.cc:492] T 29cd2c628097436db297902e5fa7c6a1 P b7ec484fb401414bb9ee34f2ec284b22: Bootstrap starting.
I20260809 04:44:31.212491 21793 tablet_bootstrap.cc:492] T 29cd2c628097436db297902e5fa7c6a1 P 65056858c7d946919f793c452292c318: Bootstrap starting.
I20260809 04:44:31.212623 21792 tablet_bootstrap.cc:654] T 29cd2c628097436db297902e5fa7c6a1 P b7ec484fb401414bb9ee34f2ec284b22: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:31.214612 21793 tablet_bootstrap.cc:654] T 29cd2c628097436db297902e5fa7c6a1 P 65056858c7d946919f793c452292c318: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:31.217367 21791 tablet_bootstrap.cc:492] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575: No bootstrap required, opened a new log
I20260809 04:44:31.217567 21791 ts_tablet_manager.cc:1397] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.000s
I20260809 04:44:31.218372 21791 raft_consensus.cc:348] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:31.218659 21791 raft_consensus.cc:374] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:31.218740 21791 raft_consensus.cc:729] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76e2a575ab054736a27c05ecfe23b575, State: Initialized, Role: FOLLOWER
I20260809 04:44:31.218920 21792 tablet_bootstrap.cc:492] T 29cd2c628097436db297902e5fa7c6a1 P b7ec484fb401414bb9ee34f2ec284b22: No bootstrap required, opened a new log
I20260809 04:44:31.219081 21792 ts_tablet_manager.cc:1397] T 29cd2c628097436db297902e5fa7c6a1 P b7ec484fb401414bb9ee34f2ec284b22: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 04:44:31.219043 21791 consensus_queue.cc:260] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:31.219522 21798 tablet_replica.cc:439] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } } }
I20260809 04:44:31.219939 21798 tablet_replica.cc:442] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:44:31.219872 21792 raft_consensus.cc:348] T 29cd2c628097436db297902e5fa7c6a1 P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:31.220165 21792 raft_consensus.cc:374] T 29cd2c628097436db297902e5fa7c6a1 P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:31.220265 21792 raft_consensus.cc:729] T 29cd2c628097436db297902e5fa7c6a1 P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7ec484fb401414bb9ee34f2ec284b22, State: Initialized, Role: FOLLOWER
I20260809 04:44:31.221009 21793 tablet_bootstrap.cc:492] T 29cd2c628097436db297902e5fa7c6a1 P 65056858c7d946919f793c452292c318: No bootstrap required, opened a new log
I20260809 04:44:31.221029 21791 ts_tablet_manager.cc:1428] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 04:44:31.221158 21793 ts_tablet_manager.cc:1397] T 29cd2c628097436db297902e5fa7c6a1 P 65056858c7d946919f793c452292c318: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.001s
I20260809 04:44:31.220602 21792 consensus_queue.cc:260] T 29cd2c628097436db297902e5fa7c6a1 P b7ec484fb401414bb9ee34f2ec284b22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:31.221753 21799 tablet_replica.cc:439] T 29cd2c628097436db297902e5fa7c6a1 P b7ec484fb401414bb9ee34f2ec284b22: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } } }
I20260809 04:44:31.221880 21793 raft_consensus.cc:348] T 29cd2c628097436db297902e5fa7c6a1 P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:31.222071 21799 tablet_replica.cc:442] T 29cd2c628097436db297902e5fa7c6a1 P b7ec484fb401414bb9ee34f2ec284b22: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:44:31.222131 21793 raft_consensus.cc:374] T 29cd2c628097436db297902e5fa7c6a1 P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:31.222237 21793 raft_consensus.cc:729] T 29cd2c628097436db297902e5fa7c6a1 P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65056858c7d946919f793c452292c318, State: Initialized, Role: FOLLOWER
I20260809 04:44:31.222493 21793 consensus_queue.cc:260] T 29cd2c628097436db297902e5fa7c6a1 P 65056858c7d946919f793c452292c318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:31.223130 21793 ts_tablet_manager.cc:1428] T 29cd2c628097436db297902e5fa7c6a1 P 65056858c7d946919f793c452292c318: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 04:44:31.223093 21800 tablet_replica.cc:439] T 29cd2c628097436db297902e5fa7c6a1 P 65056858c7d946919f793c452292c318: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } } }
I20260809 04:44:31.223352 21800 tablet_replica.cc:442] T 29cd2c628097436db297902e5fa7c6a1 P 65056858c7d946919f793c452292c318: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:44:31.223348 21792 ts_tablet_manager.cc:1428] T 29cd2c628097436db297902e5fa7c6a1 P b7ec484fb401414bb9ee34f2ec284b22: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 04:44:31.250401 21798 raft_consensus.cc:482] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:44:31.250582 21798 raft_consensus.cc:504] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:31.252027 21798 leader_election.cc:288] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 65056858c7d946919f793c452292c318 (127.12.234.1:40729), b7ec484fb401414bb9ee34f2ec284b22 (127.12.234.3:33343)
I20260809 04:44:31.259833 21789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:31.253242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:53659 (local address 127.12.234.3:33343)
0809 04:44:31.253424 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:31.253430 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:31.253753 (+ 323us) server_negotiation.cc:407] Connection header received
0809 04:44:31.253998 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:31.254003 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:31.254079 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:31.254146 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:31.254929 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.255961 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.258048 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.258861 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.259040 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:31.259110 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:31.259456 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 04:44:31.259584 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 04:44:31.259933 21786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:31.252431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:50721 (local address 127.12.234.1:40729)
0809 04:44:31.252677 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:31.252687 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:31.253255 (+ 568us) server_negotiation.cc:407] Connection header received
0809 04:44:31.253359 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:31.253365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:31.253462 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:31.253557 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:44:31.254334 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.255282 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.257513 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.258933 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.259089 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:31.259191 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:31.259565 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 04:44:31.259697 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 04:44:31.259866 21838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:31.253117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:33343 (local address 127.12.234.2:53659)
0809 04:44:31.253672 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:31.253688 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:31.253838 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:31.254333 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:31.254337 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:31.254359 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:31.254726 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:31.254737 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.256156 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.256168 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:31.257894 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:31.257905 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.259076 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.259081 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:31.259252 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:31.259311 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 04:44:31.259397 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 04:44:31.259520 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":23,"thread_start_us":1234,"threads_started":1}
I20260809 04:44:31.260587 21837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:31.252310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:40729 (local address 127.12.234.2:50721)
0809 04:44:31.252957 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:31.252972 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:31.253134 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:31.253729 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:31.253736 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:31.253771 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:44:31.254162 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:31.254174 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.255405 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.255411 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:31.257295 (+ 1884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:31.257309 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.259252 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.259257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:31.259411 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:31.259434 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 04:44:31.260068 (+ 634us) client_negotiation.cc:240] Negotiation successful
0809 04:44:31.260209 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":1,"thread_start_us":404,"threads_started":1}
I20260809 04:44:31.260766 21718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29cd2c628097436db297902e5fa7c6a1" candidate_uuid: "76e2a575ab054736a27c05ecfe23b575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ec484fb401414bb9ee34f2ec284b22" is_pre_election: true
I20260809 04:44:31.261016 21718 raft_consensus.cc:2454] T 29cd2c628097436db297902e5fa7c6a1 P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76e2a575ab054736a27c05ecfe23b575 in term 0.
I20260809 04:44:31.261101 21413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29cd2c628097436db297902e5fa7c6a1" candidate_uuid: "76e2a575ab054736a27c05ecfe23b575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65056858c7d946919f793c452292c318" is_pre_election: true
I20260809 04:44:31.261382 21413 raft_consensus.cc:2454] T 29cd2c628097436db297902e5fa7c6a1 P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76e2a575ab054736a27c05ecfe23b575 in term 0.
I20260809 04:44:31.261610 21491 leader_election.cc:302] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76e2a575ab054736a27c05ecfe23b575, b7ec484fb401414bb9ee34f2ec284b22; no voters:
I20260809 04:44:31.261994 21798 raft_consensus.cc:2781] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:44:31.262111 21798 raft_consensus.cc:482] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:44:31.262224 21798 raft_consensus.cc:3037] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:31.264279 21798 raft_consensus.cc:504] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:31.264937 21798 leader_election.cc:288] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575 [CANDIDATE]: Term 1 election: Requested vote from peers 65056858c7d946919f793c452292c318 (127.12.234.1:40729), b7ec484fb401414bb9ee34f2ec284b22 (127.12.234.3:33343)
I20260809 04:44:31.265488 21718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29cd2c628097436db297902e5fa7c6a1" candidate_uuid: "76e2a575ab054736a27c05ecfe23b575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ec484fb401414bb9ee34f2ec284b22"
I20260809 04:44:31.265517 21413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29cd2c628097436db297902e5fa7c6a1" candidate_uuid: "76e2a575ab054736a27c05ecfe23b575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65056858c7d946919f793c452292c318"
I20260809 04:44:31.265731 21718 raft_consensus.cc:3037] T 29cd2c628097436db297902e5fa7c6a1 P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:31.265766 21413 raft_consensus.cc:3037] T 29cd2c628097436db297902e5fa7c6a1 P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:31.268831 21718 raft_consensus.cc:2454] T 29cd2c628097436db297902e5fa7c6a1 P b7ec484fb401414bb9ee34f2ec284b22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76e2a575ab054736a27c05ecfe23b575 in term 1.
I20260809 04:44:31.269109 21413 raft_consensus.cc:2454] T 29cd2c628097436db297902e5fa7c6a1 P 65056858c7d946919f793c452292c318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76e2a575ab054736a27c05ecfe23b575 in term 1.
I20260809 04:44:31.269238 21491 leader_election.cc:302] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76e2a575ab054736a27c05ecfe23b575, b7ec484fb401414bb9ee34f2ec284b22; no voters:
I20260809 04:44:31.269528 21798 raft_consensus.cc:2781] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:31.270051 21798 raft_consensus.cc:686] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575 [term 1 LEADER]: Becoming Leader. State: Replica: 76e2a575ab054736a27c05ecfe23b575, State: Running, Role: LEADER
I20260809 04:44:31.270444 21798 consensus_queue.cc:237] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:31.272775 21839 tablet_replica.cc:439] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575: TxnStatusTablet state changed. Reason: New leader 76e2a575ab054736a27c05ecfe23b575. Latest consensus state: current_term: 1 leader_uuid: "76e2a575ab054736a27c05ecfe23b575" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } } }
I20260809 04:44:31.273171 21839 tablet_replica.cc:442] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:44:31.273835 21841 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 04:44:31.274080 21841 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 04:44:31.274093 21247 catalog_manager.cc:5495] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76e2a575ab054736a27c05ecfe23b575 (127.12.234.2). New cstate: current_term: 1 leader_uuid: "76e2a575ab054736a27c05ecfe23b575" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } health_report { overall_health: UNKNOWN } } }
I20260809 04:44:31.293879 21832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:31.286487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:42223 (local address 127.0.0.1:44956)
0809 04:44:31.286727 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:31.286747 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:31.286890 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:31.287317 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:31.287323 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:31.287337 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:44:31.287566 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:31.287578 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.289144 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.289152 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:31.290795 (+ 1643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:31.290811 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.291938 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.291945 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:31.292204 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:31.292277 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:44:31.293284 (+ 1007us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:44:31.293304 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 04:44:31.293380 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 04:44:31.293545 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 04:44:31.294756 21787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:31.286794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44956 (local address 127.12.234.2:42223)
0809 04:44:31.286960 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:31.286965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:31.286983 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:44:31.287054 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:31.287058 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:31.287145 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:31.287261 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:44:31.287716 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.288966 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.290969 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.291761 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.291895 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:31.292470 (+ 575us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:44:31.293091 (+ 621us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:44:31.293230 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:31.294171 (+ 941us) server_negotiation.cc:299] Negotiation successful
0809 04:44:31.294306 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 04:44:31.298826 21718 raft_consensus.cc:1261] T 29cd2c628097436db297902e5fa7c6a1 P b7ec484fb401414bb9ee34f2ec284b22 [term 1 FOLLOWER]: Refusing update from remote peer 76e2a575ab054736a27c05ecfe23b575: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:31.299204 21413 raft_consensus.cc:1261] T 29cd2c628097436db297902e5fa7c6a1 P 65056858c7d946919f793c452292c318 [term 1 FOLLOWER]: Refusing update from remote peer 76e2a575ab054736a27c05ecfe23b575: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:31.299593 21798 consensus_queue.cc:1035] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:31.299896 21839 consensus_queue.cc:1035] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:31.313105 21800 tablet_replica.cc:439] T 29cd2c628097436db297902e5fa7c6a1 P 65056858c7d946919f793c452292c318: TxnStatusTablet state changed. Reason: New leader 76e2a575ab054736a27c05ecfe23b575. Latest consensus state: current_term: 1 leader_uuid: "76e2a575ab054736a27c05ecfe23b575" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } } }
I20260809 04:44:31.313409 21800 tablet_replica.cc:442] T 29cd2c628097436db297902e5fa7c6a1 P 65056858c7d946919f793c452292c318: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:44:31.313446 21799 tablet_replica.cc:439] T 29cd2c628097436db297902e5fa7c6a1 P b7ec484fb401414bb9ee34f2ec284b22: TxnStatusTablet state changed. Reason: New leader 76e2a575ab054736a27c05ecfe23b575. Latest consensus state: current_term: 1 leader_uuid: "76e2a575ab054736a27c05ecfe23b575" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } } }
I20260809 04:44:31.313870 21799 tablet_replica.cc:442] T 29cd2c628097436db297902e5fa7c6a1 P b7ec484fb401414bb9ee34f2ec284b22: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:44:31.316742 21798 tablet_replica.cc:439] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "76e2a575ab054736a27c05ecfe23b575" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } } }
I20260809 04:44:31.317127 21798 tablet_replica.cc:442] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:44:31.317373 21808 mvcc.cc:205] Tried to move back new op lower bound from 7316482749632589824 to 7316482749527392256. Current Snapshot: MvccSnapshot[applied={T|T < 7316482749632589824 or (T in {7316482749632589824})}]
I20260809 04:44:31.318657 21839 tablet_replica.cc:439] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "76e2a575ab054736a27c05ecfe23b575" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } } }
I20260809 04:44:31.318890 21839 tablet_replica.cc:442] T 29cd2c628097436db297902e5fa7c6a1 P 76e2a575ab054736a27c05ecfe23b575: This TxnStatusTablet replica's current role is: LEADER
I20260809 04:44:31.320966 21800 tablet_replica.cc:439] T 29cd2c628097436db297902e5fa7c6a1 P 65056858c7d946919f793c452292c318: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "76e2a575ab054736a27c05ecfe23b575" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } } }
I20260809 04:44:31.321285 21800 tablet_replica.cc:442] T 29cd2c628097436db297902e5fa7c6a1 P 65056858c7d946919f793c452292c318: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:44:31.323032 21807 mvcc.cc:205] Tried to move back new op lower bound from 7316482749632589824 to 7316482749527392256. Current Snapshot: MvccSnapshot[applied={T|T < 7316482749632589824}]
I20260809 04:44:31.323921 21799 tablet_replica.cc:439] T 29cd2c628097436db297902e5fa7c6a1 P b7ec484fb401414bb9ee34f2ec284b22: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "76e2a575ab054736a27c05ecfe23b575" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } } }
I20260809 04:44:31.324373 21799 tablet_replica.cc:442] T 29cd2c628097436db297902e5fa7c6a1 P b7ec484fb401414bb9ee34f2ec284b22: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 04:44:31.335736 21789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:31.328352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46618 (local address 127.12.234.3:33343)
0809 04:44:31.328508 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:31.328514 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:31.328533 (+ 19us) server_negotiation.cc:407] Connection header received
0809 04:44:31.328750 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:31.328755 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:31.328840 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:31.328975 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:44:31.329580 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.330677 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.332715 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.333474 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.333608 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:31.334142 (+ 534us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:44:31.334855 (+ 713us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:44:31.335022 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:31.335204 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 04:44:31.335291 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:44:31.353749 21787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:31.347106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:50577 (local address 127.12.234.2:42223)
0809 04:44:31.347258 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:31.347263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:31.347370 (+ 107us) server_negotiation.cc:407] Connection header received
0809 04:44:31.347729 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:31.347735 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:31.347832 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:31.347933 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:44:31.348717 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.349861 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.352009 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.352512 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.352607 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:31.353132 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:44:31.353227 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:44:31.353292 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:31.353466 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 04:44:31.353524 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 04:44:31.354038 21848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:31.346676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:42223 (local address 127.12.234.3:50577)
0809 04:44:31.347266 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:31.347283 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:31.347520 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:31.348134 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:31.348141 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:31.348155 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:44:31.348507 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:31.348520 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.350053 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.350058 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:31.351826 (+ 1768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:31.351844 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.352688 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.352694 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:31.352868 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:31.352957 (+ 89us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:44:31.353329 (+ 372us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:44:31.353340 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 04:44:31.353434 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 04:44:31.353564 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":658,"threads_started":1}
I20260809 04:44:31.363106 21569 raft_consensus.cc:1261] T 3ea9911969704119b3b7ab2972c07682 P 76e2a575ab054736a27c05ecfe23b575 [term 1 FOLLOWER]: Refusing update from remote peer b7ec484fb401414bb9ee34f2ec284b22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:31.363163 21413 raft_consensus.cc:1261] T 3ea9911969704119b3b7ab2972c07682 P 65056858c7d946919f793c452292c318 [term 1 FOLLOWER]: Refusing update from remote peer b7ec484fb401414bb9ee34f2ec284b22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:31.363790 21824 consensus_queue.cc:1035] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:31.364141 21799 consensus_queue.cc:1035] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:31.406853 21789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:31.397657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:44455 (local address 127.12.234.3:33343)
0809 04:44:31.397830 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:31.397835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:31.400771 (+ 2936us) server_negotiation.cc:407] Connection header received
0809 04:44:31.401014 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:31.401025 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:31.401089 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:31.401196 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:44:31.401816 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.402522 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.404782 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.405492 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.405627 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:31.406142 (+ 515us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:44:31.406278 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:44:31.406383 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:31.406490 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 04:44:31.406567 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:44:31.406880 21854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:31.397566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:33343 (local address 127.12.234.2:44455)
0809 04:44:31.400669 (+ 3103us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:31.400695 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:31.400862 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:31.401212 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:31.401216 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:31.401229 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:44:31.401632 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:31.401642 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.402657 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.402665 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:31.404620 (+ 1955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:31.404635 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.405676 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.405680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:31.405879 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:31.405956 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:44:31.406396 (+ 440us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:44:31.406405 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 04:44:31.406464 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 04:44:31.406559 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2895,"thread_start_us":2693,"threads_started":1}
I20260809 04:44:31.449637 21247 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:31.449959 21247 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:31.454306 21247 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c45f58836ad348a895b951ab8de06549: Sending DeleteTablet for 3 replicas of tablet 3ea9911969704119b3b7ab2972c07682
I20260809 04:44:31.455281 21391 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ea9911969704119b3b7ab2972c07682 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:31 UTC) from {username='oryx', principal='oryx/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:37078
I20260809 04:44:31.457338 21540 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ea9911969704119b3b7ab2972c07682 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:31 UTC) from {username='oryx', principal='oryx/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:44954
I20260809 04:44:31.462790 21688 tablet_service.cc:1514] Processing DeleteTablet for tablet 3ea9911969704119b3b7ab2972c07682 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:31 UTC) from {username='oryx', principal='oryx/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:57592
I20260809 04:44:31.479821 21861 tablet_replica.cc:330] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22: stopping tablet replica
I20260809 04:44:31.480232 21861 raft_consensus.cc:2229] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:44:31.481160 21861 raft_consensus.cc:2258] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:31.483249 21861 ts_tablet_manager.cc:1905] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:31.484483 21860 tablet_replica.cc:330] T 3ea9911969704119b3b7ab2972c07682 P 76e2a575ab054736a27c05ecfe23b575: stopping tablet replica
I20260809 04:44:31.484786 21859 tablet_replica.cc:330] T 3ea9911969704119b3b7ab2972c07682 P 65056858c7d946919f793c452292c318: stopping tablet replica
I20260809 04:44:31.485075 21859 raft_consensus.cc:2229] T 3ea9911969704119b3b7ab2972c07682 P 65056858c7d946919f793c452292c318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:44:31.485445 21859 raft_consensus.cc:2258] T 3ea9911969704119b3b7ab2972c07682 P 65056858c7d946919f793c452292c318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:31.486572 21859 ts_tablet_manager.cc:1905] T 3ea9911969704119b3b7ab2972c07682 P 65056858c7d946919f793c452292c318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:31.484704 21860 raft_consensus.cc:2229] T 3ea9911969704119b3b7ab2972c07682 P 76e2a575ab054736a27c05ecfe23b575 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:44:31.487295 21860 raft_consensus.cc:2258] T 3ea9911969704119b3b7ab2972c07682 P 76e2a575ab054736a27c05ecfe23b575 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:31.488605 21860 ts_tablet_manager.cc:1905] T 3ea9911969704119b3b7ab2972c07682 P 76e2a575ab054736a27c05ecfe23b575: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:31.495954 21859 ts_tablet_manager.cc:1918] T 3ea9911969704119b3b7ab2972c07682 P 65056858c7d946919f793c452292c318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:44:31.496158 21859 log.cc:1196] T 3ea9911969704119b3b7ab2972c07682 P 65056858c7d946919f793c452292c318: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-0/wal/wals/3ea9911969704119b3b7ab2972c07682
I20260809 04:44:31.497021 21859 ts_tablet_manager.cc:1939] T 3ea9911969704119b3b7ab2972c07682 P 65056858c7d946919f793c452292c318: Deleting consensus metadata
I20260809 04:44:31.498194 21230 catalog_manager.cc:4841] TS 65056858c7d946919f793c452292c318 (127.12.234.1:40729): tablet 3ea9911969704119b3b7ab2972c07682 (table test-table [id=4f0f8324669e4e0490c7a0f0bfdc7df3]) successfully deleted
I20260809 04:44:31.505434 21861 ts_tablet_manager.cc:1918] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:44:31.505595 21861 log.cc:1196] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-2/wal/wals/3ea9911969704119b3b7ab2972c07682
I20260809 04:44:31.506028 21861 ts_tablet_manager.cc:1939] T 3ea9911969704119b3b7ab2972c07682 P b7ec484fb401414bb9ee34f2ec284b22: Deleting consensus metadata
I20260809 04:44:31.507465 21230 catalog_manager.cc:4841] TS b7ec484fb401414bb9ee34f2ec284b22 (127.12.234.3:33343): tablet 3ea9911969704119b3b7ab2972c07682 (table test-table [id=4f0f8324669e4e0490c7a0f0bfdc7df3]) successfully deleted
I20260809 04:44:31.510332 21860 ts_tablet_manager.cc:1918] T 3ea9911969704119b3b7ab2972c07682 P 76e2a575ab054736a27c05ecfe23b575: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:44:31.510571 21860 log.cc:1196] T 3ea9911969704119b3b7ab2972c07682 P 76e2a575ab054736a27c05ecfe23b575: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-1/wal/wals/3ea9911969704119b3b7ab2972c07682
I20260809 04:44:31.511093 21860 ts_tablet_manager.cc:1939] T 3ea9911969704119b3b7ab2972c07682 P 76e2a575ab054736a27c05ecfe23b575: Deleting consensus metadata
I20260809 04:44:31.512288 21229 catalog_manager.cc:4841] TS 76e2a575ab054736a27c05ecfe23b575 (127.12.234.2:42223): tablet 3ea9911969704119b3b7ab2972c07682 (table test-table [id=4f0f8324669e4e0490c7a0f0bfdc7df3]) successfully deleted
Aug 09 04:44:31 dist-test-slave-1jjb krb5kdc[21197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 04:44:31 dist-test-slave-1jjb krb5kdc[21197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:31 dist-test-slave-1jjb krb5kdc[21197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 04:44:31.571094 21831 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:31.557696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57346 (local address 127.12.234.62:33429)
0809 04:44:31.557887 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:31.557895 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:31.558311 (+ 416us) server_negotiation.cc:407] Connection header received
0809 04:44:31.558493 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:31.558499 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:31.558606 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:31.558741 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:31.560315 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.561604 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.563290 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.563951 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.564099 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:31.570829 (+ 6730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57346: BlockingRecv error: recv got EOF from 127.0.0.1:57346 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 09 04:44:31 dist-test-slave-1jjb krb5kdc[21197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM
I20260809 04:44:31.596093 21831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:31.578118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57352 (local address 127.12.234.62:33429)
0809 04:44:31.578275 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:31.578280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:31.578474 (+ 194us) server_negotiation.cc:407] Connection header received
0809 04:44:31.578736 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:31.578741 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:31.578811 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:31.578887 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:31.580470 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.581506 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.583205 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.583935 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.584029 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:31.589116 (+ 5087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:31.589123 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:31.589128 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:31.589164 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:31.592422 (+ 3258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:31.593190 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:31.593197 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:31.593203 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:31.593369 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:31.593818 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:31.593823 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:31.593826 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:31.594251 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:31.594563 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:31.595262 (+ 699us) server_negotiation.cc:299] Negotiation successful
0809 04:44:31.595620 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 04:44:31.601063 21247 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:44:31.601683 21247 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:44:31.610272 21540 tablet_service.cc:1467] Processing CreateTablet for tablet 4bb8a1029a4842e8941867b653521569 (DEFAULT_TABLE table=test-table [id=6fb32699005d45b2bde2118d9ac870fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:31.610761 21540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bb8a1029a4842e8941867b653521569. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:31.610692 21391 tablet_service.cc:1467] Processing CreateTablet for tablet 4bb8a1029a4842e8941867b653521569 (DEFAULT_TABLE table=test-table [id=6fb32699005d45b2bde2118d9ac870fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:31.611114 21391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bb8a1029a4842e8941867b653521569. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:31.612447 21688 tablet_service.cc:1467] Processing CreateTablet for tablet 4bb8a1029a4842e8941867b653521569 (DEFAULT_TABLE table=test-table [id=6fb32699005d45b2bde2118d9ac870fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:31.612836 21688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bb8a1029a4842e8941867b653521569. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:31.620317 21791 tablet_bootstrap.cc:492] T 4bb8a1029a4842e8941867b653521569 P 76e2a575ab054736a27c05ecfe23b575: Bootstrap starting.
I20260809 04:44:31.621992 21791 tablet_bootstrap.cc:654] T 4bb8a1029a4842e8941867b653521569 P 76e2a575ab054736a27c05ecfe23b575: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:31.626437 21793 tablet_bootstrap.cc:492] T 4bb8a1029a4842e8941867b653521569 P 65056858c7d946919f793c452292c318: Bootstrap starting.
I20260809 04:44:31.628084 21793 tablet_bootstrap.cc:654] T 4bb8a1029a4842e8941867b653521569 P 65056858c7d946919f793c452292c318: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:31.629218 21792 tablet_bootstrap.cc:492] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22: Bootstrap starting.
I20260809 04:44:31.630883 21792 tablet_bootstrap.cc:654] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:31.639192 21791 tablet_bootstrap.cc:492] T 4bb8a1029a4842e8941867b653521569 P 76e2a575ab054736a27c05ecfe23b575: No bootstrap required, opened a new log
I20260809 04:44:31.639397 21791 ts_tablet_manager.cc:1397] T 4bb8a1029a4842e8941867b653521569 P 76e2a575ab054736a27c05ecfe23b575: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.005s
I20260809 04:44:31.640352 21791 raft_consensus.cc:348] T 4bb8a1029a4842e8941867b653521569 P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:31.640748 21791 raft_consensus.cc:374] T 4bb8a1029a4842e8941867b653521569 P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:31.640900 21791 raft_consensus.cc:729] T 4bb8a1029a4842e8941867b653521569 P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76e2a575ab054736a27c05ecfe23b575, State: Initialized, Role: FOLLOWER
I20260809 04:44:31.641176 21791 consensus_queue.cc:260] T 4bb8a1029a4842e8941867b653521569 P 76e2a575ab054736a27c05ecfe23b575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:31.641826 21791 ts_tablet_manager.cc:1428] T 4bb8a1029a4842e8941867b653521569 P 76e2a575ab054736a27c05ecfe23b575: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 04:44:31.649694 21792 tablet_bootstrap.cc:492] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22: No bootstrap required, opened a new log
I20260809 04:44:31.649866 21792 ts_tablet_manager.cc:1397] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22: Time spent bootstrapping tablet: real 0.021s user 0.009s sys 0.001s
I20260809 04:44:31.650705 21792 raft_consensus.cc:348] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:31.650974 21792 raft_consensus.cc:374] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:31.651059 21792 raft_consensus.cc:729] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7ec484fb401414bb9ee34f2ec284b22, State: Initialized, Role: FOLLOWER
I20260809 04:44:31.651628 21792 consensus_queue.cc:260] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:31.652379 21792 ts_tablet_manager.cc:1428] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260809 04:44:31.654410 21793 tablet_bootstrap.cc:492] T 4bb8a1029a4842e8941867b653521569 P 65056858c7d946919f793c452292c318: No bootstrap required, opened a new log
I20260809 04:44:31.654575 21793 ts_tablet_manager.cc:1397] T 4bb8a1029a4842e8941867b653521569 P 65056858c7d946919f793c452292c318: Time spent bootstrapping tablet: real 0.028s user 0.003s sys 0.007s
I20260809 04:44:31.656123 21793 raft_consensus.cc:348] T 4bb8a1029a4842e8941867b653521569 P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:31.656383 21793 raft_consensus.cc:374] T 4bb8a1029a4842e8941867b653521569 P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:31.656468 21793 raft_consensus.cc:729] T 4bb8a1029a4842e8941867b653521569 P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65056858c7d946919f793c452292c318, State: Initialized, Role: FOLLOWER
I20260809 04:44:31.656728 21793 consensus_queue.cc:260] T 4bb8a1029a4842e8941867b653521569 P 65056858c7d946919f793c452292c318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:31.657362 21793 ts_tablet_manager.cc:1428] T 4bb8a1029a4842e8941867b653521569 P 65056858c7d946919f793c452292c318: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 04:44:31.755203 21824 raft_consensus.cc:482] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:44:31.755465 21824 raft_consensus.cc:504] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:31.756326 21824 leader_election.cc:288] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 65056858c7d946919f793c452292c318 (127.12.234.1:40729), 76e2a575ab054736a27c05ecfe23b575 (127.12.234.2:42223)
I20260809 04:44:31.756946 21413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bb8a1029a4842e8941867b653521569" candidate_uuid: "b7ec484fb401414bb9ee34f2ec284b22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65056858c7d946919f793c452292c318" is_pre_election: true
I20260809 04:44:31.757048 21569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bb8a1029a4842e8941867b653521569" candidate_uuid: "b7ec484fb401414bb9ee34f2ec284b22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76e2a575ab054736a27c05ecfe23b575" is_pre_election: true
I20260809 04:44:31.757181 21413 raft_consensus.cc:2454] T 4bb8a1029a4842e8941867b653521569 P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7ec484fb401414bb9ee34f2ec284b22 in term 0.
I20260809 04:44:31.757278 21569 raft_consensus.cc:2454] T 4bb8a1029a4842e8941867b653521569 P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7ec484fb401414bb9ee34f2ec284b22 in term 0.
I20260809 04:44:31.757774 21638 leader_election.cc:302] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76e2a575ab054736a27c05ecfe23b575, b7ec484fb401414bb9ee34f2ec284b22; no voters:
I20260809 04:44:31.758199 21824 raft_consensus.cc:2781] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:44:31.758309 21824 raft_consensus.cc:482] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:44:31.758426 21824 raft_consensus.cc:3037] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:31.761618 21824 raft_consensus.cc:504] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:31.762543 21569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bb8a1029a4842e8941867b653521569" candidate_uuid: "b7ec484fb401414bb9ee34f2ec284b22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76e2a575ab054736a27c05ecfe23b575"
I20260809 04:44:31.762619 21413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bb8a1029a4842e8941867b653521569" candidate_uuid: "b7ec484fb401414bb9ee34f2ec284b22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65056858c7d946919f793c452292c318"
I20260809 04:44:31.762727 21569 raft_consensus.cc:3037] T 4bb8a1029a4842e8941867b653521569 P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:31.762815 21413 raft_consensus.cc:3037] T 4bb8a1029a4842e8941867b653521569 P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:31.765331 21569 raft_consensus.cc:2454] T 4bb8a1029a4842e8941867b653521569 P 76e2a575ab054736a27c05ecfe23b575 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7ec484fb401414bb9ee34f2ec284b22 in term 1.
I20260809 04:44:31.765683 21413 raft_consensus.cc:2454] T 4bb8a1029a4842e8941867b653521569 P 65056858c7d946919f793c452292c318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7ec484fb401414bb9ee34f2ec284b22 in term 1.
I20260809 04:44:31.765722 21638 leader_election.cc:302] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76e2a575ab054736a27c05ecfe23b575, b7ec484fb401414bb9ee34f2ec284b22; no voters:
I20260809 04:44:31.765996 21855 raft_consensus.cc:2781] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:31.766136 21855 raft_consensus.cc:686] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22 [term 1 LEADER]: Becoming Leader. State: Replica: b7ec484fb401414bb9ee34f2ec284b22, State: Running, Role: LEADER
I20260809 04:44:31.766490 21824 leader_election.cc:288] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22 [CANDIDATE]: Term 1 election: Requested vote from peers 65056858c7d946919f793c452292c318 (127.12.234.1:40729), 76e2a575ab054736a27c05ecfe23b575 (127.12.234.2:42223)
I20260809 04:44:31.766438 21855 consensus_queue.cc:237] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:31.769538 21245 catalog_manager.cc:5495] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7ec484fb401414bb9ee34f2ec284b22 (127.12.234.3). New cstate: current_term: 1 leader_uuid: "b7ec484fb401414bb9ee34f2ec284b22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } health_report { overall_health: HEALTHY } } }
I20260809 04:44:31.820602 21789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:31.815216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46620 (local address 127.12.234.3:33343)
0809 04:44:31.815434 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:31.815441 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:31.815481 (+ 40us) server_negotiation.cc:407] Connection header received
0809 04:44:31.815555 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:31.815561 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:31.815658 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:31.815766 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:44:31.816106 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.817232 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.818663 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:31.819283 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:31.819439 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:31.819832 (+ 393us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:44:31.819945 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:44:31.820049 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:31.820225 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 04:44:31.820304 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 04:44:31.823446 21569 raft_consensus.cc:1261] T 4bb8a1029a4842e8941867b653521569 P 76e2a575ab054736a27c05ecfe23b575 [term 1 FOLLOWER]: Refusing update from remote peer b7ec484fb401414bb9ee34f2ec284b22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:31.824111 21413 raft_consensus.cc:1261] T 4bb8a1029a4842e8941867b653521569 P 65056858c7d946919f793c452292c318 [term 1 FOLLOWER]: Refusing update from remote peer b7ec484fb401414bb9ee34f2ec284b22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:31.824035 21855 consensus_queue.cc:1035] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:31.824709 21855 consensus_queue.cc:1035] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:31.835222 21246 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:31.835467 21246 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:31.839779 21246 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c45f58836ad348a895b951ab8de06549: Sending DeleteTablet for 3 replicas of tablet 4bb8a1029a4842e8941867b653521569
I20260809 04:44:31.840651 21391 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bb8a1029a4842e8941867b653521569 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:31 UTC) from {username='oryx', principal='oryx/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:37078
I20260809 04:44:31.840826 21540 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bb8a1029a4842e8941867b653521569 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:31 UTC) from {username='oryx', principal='oryx/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:44954
I20260809 04:44:31.840948 21859 tablet_replica.cc:330] T 4bb8a1029a4842e8941867b653521569 P 65056858c7d946919f793c452292c318: stopping tablet replica
I20260809 04:44:31.841133 21859 raft_consensus.cc:2229] T 4bb8a1029a4842e8941867b653521569 P 65056858c7d946919f793c452292c318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:44:31.841148 21688 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bb8a1029a4842e8941867b653521569 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:31 UTC) from {username='oryx', principal='oryx/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:57592
I20260809 04:44:31.841356 21859 raft_consensus.cc:2258] T 4bb8a1029a4842e8941867b653521569 P 65056858c7d946919f793c452292c318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:31.841442 21860 tablet_replica.cc:330] T 4bb8a1029a4842e8941867b653521569 P 76e2a575ab054736a27c05ecfe23b575: stopping tablet replica
I20260809 04:44:31.841641 21860 raft_consensus.cc:2229] T 4bb8a1029a4842e8941867b653521569 P 76e2a575ab054736a27c05ecfe23b575 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:44:31.841825 21860 raft_consensus.cc:2258] T 4bb8a1029a4842e8941867b653521569 P 76e2a575ab054736a27c05ecfe23b575 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:31.842068 21861 tablet_replica.cc:330] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22: stopping tablet replica
I20260809 04:44:31.842286 21861 raft_consensus.cc:2229] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:44:31.842569 21861 raft_consensus.cc:2258] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:31.842559 21247 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:44:31.843120 21247 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:44:31.843536 21688 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bb8a1029a4842e8941867b653521569 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:31 UTC) from {username='oryx', principal='oryx/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:57592
I20260809 04:44:31.843641 21859 ts_tablet_manager.cc:1905] T 4bb8a1029a4842e8941867b653521569 P 65056858c7d946919f793c452292c318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:31.844545 21860 ts_tablet_manager.cc:1905] T 4bb8a1029a4842e8941867b653521569 P 76e2a575ab054736a27c05ecfe23b575: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:31.850765 21860 ts_tablet_manager.cc:1918] T 4bb8a1029a4842e8941867b653521569 P 76e2a575ab054736a27c05ecfe23b575: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:31.850975 21860 log.cc:1196] T 4bb8a1029a4842e8941867b653521569 P 76e2a575ab054736a27c05ecfe23b575: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-1/wal/wals/4bb8a1029a4842e8941867b653521569
I20260809 04:44:31.851506 21860 ts_tablet_manager.cc:1939] T 4bb8a1029a4842e8941867b653521569 P 76e2a575ab054736a27c05ecfe23b575: Deleting consensus metadata
I20260809 04:44:31.851948 21391 tablet_service.cc:1467] Processing CreateTablet for tablet 0367af782a9641219bfc39dd873a822f (DEFAULT_TABLE table=test-table [id=fb0febbb00cb497293fe72971f005229]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:31.852110 21688 tablet_service.cc:1467] Processing CreateTablet for tablet 0367af782a9641219bfc39dd873a822f (DEFAULT_TABLE table=test-table [id=fb0febbb00cb497293fe72971f005229]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:31.852356 21391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0367af782a9641219bfc39dd873a822f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:31.852463 21688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0367af782a9641219bfc39dd873a822f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:31.856419 21540 tablet_service.cc:1467] Processing CreateTablet for tablet 0367af782a9641219bfc39dd873a822f (DEFAULT_TABLE table=test-table [id=fb0febbb00cb497293fe72971f005229]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:44:31.856781 21540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0367af782a9641219bfc39dd873a822f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:31.857939 21859 ts_tablet_manager.cc:1918] T 4bb8a1029a4842e8941867b653521569 P 65056858c7d946919f793c452292c318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:31.858091 21859 log.cc:1196] T 4bb8a1029a4842e8941867b653521569 P 65056858c7d946919f793c452292c318: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-0/wal/wals/4bb8a1029a4842e8941867b653521569
I20260809 04:44:31.858467 21793 tablet_bootstrap.cc:492] T 0367af782a9641219bfc39dd873a822f P 65056858c7d946919f793c452292c318: Bootstrap starting.
I20260809 04:44:31.858551 21859 ts_tablet_manager.cc:1939] T 4bb8a1029a4842e8941867b653521569 P 65056858c7d946919f793c452292c318: Deleting consensus metadata
I20260809 04:44:31.860152 21793 tablet_bootstrap.cc:654] T 0367af782a9641219bfc39dd873a822f P 65056858c7d946919f793c452292c318: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:31.860752 21230 catalog_manager.cc:4841] TS 65056858c7d946919f793c452292c318 (127.12.234.1:40729): tablet 4bb8a1029a4842e8941867b653521569 (table test-table [id=6fb32699005d45b2bde2118d9ac870fb]) successfully deleted
I20260809 04:44:31.862841 21791 tablet_bootstrap.cc:492] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575: Bootstrap starting.
I20260809 04:44:31.864454 21791 tablet_bootstrap.cc:654] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:31.864737 21792 tablet_bootstrap.cc:492] T 0367af782a9641219bfc39dd873a822f P b7ec484fb401414bb9ee34f2ec284b22: Bootstrap starting.
I20260809 04:44:31.866665 21792 tablet_bootstrap.cc:654] T 0367af782a9641219bfc39dd873a822f P b7ec484fb401414bb9ee34f2ec284b22: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:31.870842 21229 catalog_manager.cc:4841] TS 76e2a575ab054736a27c05ecfe23b575 (127.12.234.2:42223): tablet 4bb8a1029a4842e8941867b653521569 (table test-table [id=6fb32699005d45b2bde2118d9ac870fb]) successfully deleted
I20260809 04:44:31.874197 21861 ts_tablet_manager.cc:1905] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:31.883318 21861 ts_tablet_manager.cc:1918] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:44:31.883869 21861 log.cc:1196] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-2/wal/wals/4bb8a1029a4842e8941867b653521569
I20260809 04:44:31.884836 21792 tablet_bootstrap.cc:492] T 0367af782a9641219bfc39dd873a822f P b7ec484fb401414bb9ee34f2ec284b22: No bootstrap required, opened a new log
I20260809 04:44:31.885143 21792 ts_tablet_manager.cc:1397] T 0367af782a9641219bfc39dd873a822f P b7ec484fb401414bb9ee34f2ec284b22: Time spent bootstrapping tablet: real 0.020s user 0.004s sys 0.003s
I20260809 04:44:31.886022 21792 raft_consensus.cc:348] T 0367af782a9641219bfc39dd873a822f P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:31.886452 21792 raft_consensus.cc:374] T 0367af782a9641219bfc39dd873a822f P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:31.886590 21792 raft_consensus.cc:729] T 0367af782a9641219bfc39dd873a822f P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7ec484fb401414bb9ee34f2ec284b22, State: Initialized, Role: FOLLOWER
I20260809 04:44:31.887051 21792 consensus_queue.cc:260] T 0367af782a9641219bfc39dd873a822f P b7ec484fb401414bb9ee34f2ec284b22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:31.887853 21792 ts_tablet_manager.cc:1428] T 0367af782a9641219bfc39dd873a822f P b7ec484fb401414bb9ee34f2ec284b22: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 04:44:31.888752 21861 ts_tablet_manager.cc:1939] T 4bb8a1029a4842e8941867b653521569 P b7ec484fb401414bb9ee34f2ec284b22: Deleting consensus metadata
I20260809 04:44:31.890141 21230 catalog_manager.cc:4841] TS b7ec484fb401414bb9ee34f2ec284b22 (127.12.234.3:33343): tablet 4bb8a1029a4842e8941867b653521569 (table test-table [id=6fb32699005d45b2bde2118d9ac870fb]) successfully deleted
W20260809 04:44:31.891330 21230 catalog_manager.cc:4808] TS b7ec484fb401414bb9ee34f2ec284b22 (127.12.234.3:33343): delete failed for tablet 4bb8a1029a4842e8941867b653521569 because the tablet was not found. No further retry: Not found: Tablet not found: 4bb8a1029a4842e8941867b653521569
I20260809 04:44:31.893132 21791 tablet_bootstrap.cc:492] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575: No bootstrap required, opened a new log
I20260809 04:44:31.893321 21791 ts_tablet_manager.cc:1397] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575: Time spent bootstrapping tablet: real 0.031s user 0.014s sys 0.010s
I20260809 04:44:31.893498 21793 tablet_bootstrap.cc:492] T 0367af782a9641219bfc39dd873a822f P 65056858c7d946919f793c452292c318: No bootstrap required, opened a new log
I20260809 04:44:31.893673 21793 ts_tablet_manager.cc:1397] T 0367af782a9641219bfc39dd873a822f P 65056858c7d946919f793c452292c318: Time spent bootstrapping tablet: real 0.035s user 0.004s sys 0.024s
I20260809 04:44:31.894057 21791 raft_consensus.cc:348] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:31.894270 21791 raft_consensus.cc:374] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:31.894344 21791 raft_consensus.cc:729] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76e2a575ab054736a27c05ecfe23b575, State: Initialized, Role: FOLLOWER
I20260809 04:44:31.894572 21791 consensus_queue.cc:260] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:31.894564 21793 raft_consensus.cc:348] T 0367af782a9641219bfc39dd873a822f P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:31.895052 21791 ts_tablet_manager.cc:1428] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260809 04:44:31.895438 21793 raft_consensus.cc:374] T 0367af782a9641219bfc39dd873a822f P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:31.895529 21793 raft_consensus.cc:729] T 0367af782a9641219bfc39dd873a822f P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65056858c7d946919f793c452292c318, State: Initialized, Role: FOLLOWER
I20260809 04:44:31.896706 21793 consensus_queue.cc:260] T 0367af782a9641219bfc39dd873a822f P 65056858c7d946919f793c452292c318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:31.899467 21793 ts_tablet_manager.cc:1428] T 0367af782a9641219bfc39dd873a822f P 65056858c7d946919f793c452292c318: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20260809 04:44:32.134956 21849 raft_consensus.cc:482] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:44:32.135164 21849 raft_consensus.cc:504] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:32.135865 21849 leader_election.cc:288] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 65056858c7d946919f793c452292c318 (127.12.234.1:40729), b7ec484fb401414bb9ee34f2ec284b22 (127.12.234.3:33343)
I20260809 04:44:32.136430 21413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0367af782a9641219bfc39dd873a822f" candidate_uuid: "76e2a575ab054736a27c05ecfe23b575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65056858c7d946919f793c452292c318" is_pre_election: true
I20260809 04:44:32.136512 21718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0367af782a9641219bfc39dd873a822f" candidate_uuid: "76e2a575ab054736a27c05ecfe23b575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ec484fb401414bb9ee34f2ec284b22" is_pre_election: true
I20260809 04:44:32.136715 21413 raft_consensus.cc:2454] T 0367af782a9641219bfc39dd873a822f P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76e2a575ab054736a27c05ecfe23b575 in term 0.
I20260809 04:44:32.136807 21718 raft_consensus.cc:2454] T 0367af782a9641219bfc39dd873a822f P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76e2a575ab054736a27c05ecfe23b575 in term 0.
I20260809 04:44:32.137202 21491 leader_election.cc:302] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65056858c7d946919f793c452292c318, 76e2a575ab054736a27c05ecfe23b575; no voters:
I20260809 04:44:32.137483 21849 raft_consensus.cc:2781] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:44:32.137639 21849 raft_consensus.cc:482] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:44:32.137743 21849 raft_consensus.cc:3037] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:32.140002 21849 raft_consensus.cc:504] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:32.140686 21849 leader_election.cc:288] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575 [CANDIDATE]: Term 1 election: Requested vote from peers 65056858c7d946919f793c452292c318 (127.12.234.1:40729), b7ec484fb401414bb9ee34f2ec284b22 (127.12.234.3:33343)
I20260809 04:44:32.141117 21413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0367af782a9641219bfc39dd873a822f" candidate_uuid: "76e2a575ab054736a27c05ecfe23b575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65056858c7d946919f793c452292c318"
I20260809 04:44:32.141170 21718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0367af782a9641219bfc39dd873a822f" candidate_uuid: "76e2a575ab054736a27c05ecfe23b575" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ec484fb401414bb9ee34f2ec284b22"
I20260809 04:44:32.141289 21413 raft_consensus.cc:3037] T 0367af782a9641219bfc39dd873a822f P 65056858c7d946919f793c452292c318 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:32.141399 21718 raft_consensus.cc:3037] T 0367af782a9641219bfc39dd873a822f P b7ec484fb401414bb9ee34f2ec284b22 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:32.143569 21413 raft_consensus.cc:2454] T 0367af782a9641219bfc39dd873a822f P 65056858c7d946919f793c452292c318 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76e2a575ab054736a27c05ecfe23b575 in term 1.
I20260809 04:44:32.143961 21491 leader_election.cc:302] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65056858c7d946919f793c452292c318, 76e2a575ab054736a27c05ecfe23b575; no voters:
I20260809 04:44:32.144032 21718 raft_consensus.cc:2454] T 0367af782a9641219bfc39dd873a822f P b7ec484fb401414bb9ee34f2ec284b22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76e2a575ab054736a27c05ecfe23b575 in term 1.
I20260809 04:44:32.144273 21849 raft_consensus.cc:2781] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:32.144649 21849 raft_consensus.cc:686] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575 [term 1 LEADER]: Becoming Leader. State: Replica: 76e2a575ab054736a27c05ecfe23b575, State: Running, Role: LEADER
I20260809 04:44:32.144934 21849 consensus_queue.cc:237] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } }
I20260809 04:44:32.147243 21244 catalog_manager.cc:5495] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76e2a575ab054736a27c05ecfe23b575 (127.12.234.2). New cstate: current_term: 1 leader_uuid: "76e2a575ab054736a27c05ecfe23b575" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76e2a575ab054736a27c05ecfe23b575" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 } health_report { overall_health: UNKNOWN } } }
I20260809 04:44:32.187440 21888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:32.171526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53628 (local address 127.12.234.2:42223)
0809 04:44:32.182183 (+ 10657us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:32.182188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:32.182204 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:44:32.182259 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:32.182263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:32.182335 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:32.182426 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:44:32.182879 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:32.184036 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:32.185291 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:32.185944 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:32.186059 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:32.186648 (+ 589us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:44:32.186772 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:44:32.186856 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:32.187007 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 04:44:32.187085 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10556,"thread_start_us":10455,"threads_started":1}
I20260809 04:44:32.201522 21892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:32.195939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:42223 (local address 127.12.234.2:37885)
0809 04:44:32.196654 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:32.196668 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:32.196761 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:32.197032 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:32.197035 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:32.197043 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 04:44:32.197265 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:32.197271 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:32.198633 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:32.198637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:32.199606 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:32.199617 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:32.200539 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:32.200544 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:32.200647 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:32.200679 (+ 32us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 04:44:32.201054 (+ 375us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 04:44:32.201063 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 04:44:32.201124 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 04:44:32.201216 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":469,"threads_started":1}
I20260809 04:44:32.201529 21888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:32.195951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:37885 (local address 127.12.234.2:42223)
0809 04:44:32.196144 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:32.196150 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:32.196739 (+ 589us) server_negotiation.cc:407] Connection header received
0809 04:44:32.196856 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:32.196861 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:32.196929 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:32.197011 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:44:32.197412 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:32.198484 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:32.199737 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:32.200361 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:32.200491 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:32.200824 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:44:32.200952 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:44:32.201030 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:32.201166 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 04:44:32.201234 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":45}
I20260809 04:44:32.209249 21718 raft_consensus.cc:1261] T 0367af782a9641219bfc39dd873a822f P b7ec484fb401414bb9ee34f2ec284b22 [term 1 FOLLOWER]: Refusing update from remote peer 76e2a575ab054736a27c05ecfe23b575: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:32.209380 21413 raft_consensus.cc:1261] T 0367af782a9641219bfc39dd873a822f P 65056858c7d946919f793c452292c318 [term 1 FOLLOWER]: Refusing update from remote peer 76e2a575ab054736a27c05ecfe23b575: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:44:32.209805 21887 consensus_queue.cc:1035] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ec484fb401414bb9ee34f2ec284b22" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:32.210227 21885 consensus_queue.cc:1035] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65056858c7d946919f793c452292c318" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 40729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:44:32.257864 21244 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:32.258136 21244 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:44:32.260880 21244 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c45f58836ad348a895b951ab8de06549: Sending DeleteTablet for 3 replicas of tablet 0367af782a9641219bfc39dd873a822f
I20260809 04:44:32.261631 21540 tablet_service.cc:1514] Processing DeleteTablet for tablet 0367af782a9641219bfc39dd873a822f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:32 UTC) from {username='oryx', principal='oryx/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:44954
I20260809 04:44:32.261633 21391 tablet_service.cc:1514] Processing DeleteTablet for tablet 0367af782a9641219bfc39dd873a822f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:32 UTC) from {username='oryx', principal='oryx/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:37078
I20260809 04:44:32.261899 21859 tablet_replica.cc:330] T 0367af782a9641219bfc39dd873a822f P 65056858c7d946919f793c452292c318: stopping tablet replica
I20260809 04:44:32.261919 21860 tablet_replica.cc:330] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575: stopping tablet replica
I20260809 04:44:32.261912 21688 tablet_service.cc:1514] Processing DeleteTablet for tablet 0367af782a9641219bfc39dd873a822f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:44:32 UTC) from {username='oryx', principal='oryx/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:57592
I20260809 04:44:32.262061 21859 raft_consensus.cc:2229] T 0367af782a9641219bfc39dd873a822f P 65056858c7d946919f793c452292c318 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:44:32.262084 21860 raft_consensus.cc:2229] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:44:32.262159 21861 tablet_replica.cc:330] T 0367af782a9641219bfc39dd873a822f P b7ec484fb401414bb9ee34f2ec284b22: stopping tablet replica
I20260809 04:44:32.262233 21859 raft_consensus.cc:2258] T 0367af782a9641219bfc39dd873a822f P 65056858c7d946919f793c452292c318 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:32.262307 21861 raft_consensus.cc:2229] T 0367af782a9641219bfc39dd873a822f P b7ec484fb401414bb9ee34f2ec284b22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:44:32.262511 21861 raft_consensus.cc:2258] T 0367af782a9641219bfc39dd873a822f P b7ec484fb401414bb9ee34f2ec284b22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:32.262629 21860 raft_consensus.cc:2258] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:44:32.263489 21861 ts_tablet_manager.cc:1905] T 0367af782a9641219bfc39dd873a822f P b7ec484fb401414bb9ee34f2ec284b22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:32.263607 21860 ts_tablet_manager.cc:1905] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:32.263741 21859 ts_tablet_manager.cc:1905] T 0367af782a9641219bfc39dd873a822f P 65056858c7d946919f793c452292c318: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:44:32.264372 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 21321
I20260809 04:44:32.270323 21860 ts_tablet_manager.cc:1918] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:44:32.270469 21860 log.cc:1196] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-1/wal/wals/0367af782a9641219bfc39dd873a822f
I20260809 04:44:32.270841 21860 ts_tablet_manager.cc:1939] T 0367af782a9641219bfc39dd873a822f P 76e2a575ab054736a27c05ecfe23b575: Deleting consensus metadata
I20260809 04:44:32.271667 21859 ts_tablet_manager.cc:1918] T 0367af782a9641219bfc39dd873a822f P 65056858c7d946919f793c452292c318: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:44:32.271785 21229 catalog_manager.cc:4841] TS 76e2a575ab054736a27c05ecfe23b575 (127.12.234.2:42223): tablet 0367af782a9641219bfc39dd873a822f (table test-table [id=fb0febbb00cb497293fe72971f005229]) successfully deleted
I20260809 04:44:32.271814 21859 log.cc:1196] T 0367af782a9641219bfc39dd873a822f P 65056858c7d946919f793c452292c318: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-0/wal/wals/0367af782a9641219bfc39dd873a822f
I20260809 04:44:32.272737 21859 ts_tablet_manager.cc:1939] T 0367af782a9641219bfc39dd873a822f P 65056858c7d946919f793c452292c318: Deleting consensus metadata
I20260809 04:44:32.273897 21230 catalog_manager.cc:4841] TS 65056858c7d946919f793c452292c318 (127.12.234.1:40729): tablet 0367af782a9641219bfc39dd873a822f (table test-table [id=fb0febbb00cb497293fe72971f005229]) successfully deleted
I20260809 04:44:32.274598 21861 ts_tablet_manager.cc:1918] T 0367af782a9641219bfc39dd873a822f P b7ec484fb401414bb9ee34f2ec284b22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 04:44:32.274797 21861 log.cc:1196] T 0367af782a9641219bfc39dd873a822f P b7ec484fb401414bb9ee34f2ec284b22: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786250603326224-13224-0/minicluster-data/ts-2/wal/wals/0367af782a9641219bfc39dd873a822f
I20260809 04:44:32.275187 21861 ts_tablet_manager.cc:1939] T 0367af782a9641219bfc39dd873a822f P b7ec484fb401414bb9ee34f2ec284b22: Deleting consensus metadata
I20260809 04:44:32.275964 21230 catalog_manager.cc:4841] TS b7ec484fb401414bb9ee34f2ec284b22 (127.12.234.3:33343): tablet 0367af782a9641219bfc39dd873a822f (table test-table [id=fb0febbb00cb497293fe72971f005229]) successfully deleted
Aug 09 04:44:32 dist-test-slave-1jjb krb5kdc[21197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.234.1@KRBTEST.COM
I20260809 04:44:32.289739 21898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:32.266412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49578 (local address 127.12.234.1:40729)
0809 04:44:32.274956 (+ 8544us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:32.274963 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:32.274981 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:44:32.275036 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:32.275043 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:32.275117 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:32.275223 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:32.276723 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:32.277375 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:32.279153 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:32.279535 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:32.279620 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:32.283861 (+ 4241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:32.283874 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:32.283888 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:32.283914 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:32.286983 (+ 3069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:32.287653 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:32.287661 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:32.287664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:32.287780 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:32.288198 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:32.288205 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:32.288208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:32.288547 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:32.288867 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:32.289181 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 04:44:32.289422 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8423,"thread_start_us":8334,"threads_started":1}
I20260809 04:44:32.290355 21462 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:32.487259 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 21321
I20260809 04:44:32.515913 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 21473
Aug 09 04:44:32 dist-test-slave-1jjb krb5kdc[21197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.234.2@KRBTEST.COM
I20260809 04:44:32.535142 21888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:32.516948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53636 (local address 127.12.234.2:42223)
0809 04:44:32.517126 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:32.517133 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:32.517167 (+ 34us) server_negotiation.cc:407] Connection header received
0809 04:44:32.517249 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:32.517253 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:32.517313 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:32.517396 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:32.519171 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:32.520044 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:32.521963 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:32.522359 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:32.522454 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:32.527327 (+ 4873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:32.527343 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:32.527363 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:32.527407 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:32.531797 (+ 4390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:32.532746 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:32.532757 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:32.532762 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:32.532979 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:32.533345 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:32.533350 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:32.533354 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:32.533864 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:32.534250 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:32.534436 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 04:44:32.534686 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 04:44:32.535938 21611 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:32.679908 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 21473
I20260809 04:44:32.708742 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 21620
Aug 09 04:44:32 dist-test-slave-1jjb krb5kdc[21197](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.234.3@KRBTEST.COM
I20260809 04:44:32.724917 21901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:32.709543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46624 (local address 127.12.234.3:33343)
0809 04:44:32.710028 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:32.710033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:32.710049 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:44:32.710106 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:32.710110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:32.710170 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:32.710269 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:32.711562 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:32.712267 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:32.713492 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:32.713904 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:32.713991 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:32.718900 (+ 4909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:32.718917 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:32.718934 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:32.718967 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:32.721946 (+ 2979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:32.722630 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:32.722638 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:32.722640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:32.722773 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:32.723204 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:32.723210 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:32.723216 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:32.723595 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:32.723946 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:32.724249 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 04:44:32.724507 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":240,"threads_started":1}
I20260809 04:44:32.725600 21760 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:32.847818 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 21620
I20260809 04:44:32.873332 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 21210
I20260809 04:44:32.874541 21288 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:32.988310 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 21210
2026-08-09T04:44:33Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7150 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:33 dist-test-slave-1jjb krb5kdc[21906](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:33 dist-test-slave-1jjb krb5kdc[21906](info): set up 2 sockets
Aug 09 04:44:33 dist-test-slave-1jjb krb5kdc[21906](info): commencing operation
krb5kdc: starting...
W20260809 04:44:34.101061 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:44:34 dist-test-slave-1jjb krb5kdc[21906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:44:34.327664 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:45173
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:35431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:45173
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:34.649497 21919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:34.649801 21919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:34.649953 21919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:34.656479 21919 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:34.656596 21919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:34.656684 21919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:34.656769 21919 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:34.656847 21919 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:34.665693 21919 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:35431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:45173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:45173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:34.666823 21919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:34.668155 21919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:34.680258 21924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:34.680434 21927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:34.680117 21925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:34.681767 21919 server_base.cc:1034] running on GCE node
I20260809 04:44:34.682463 21919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:34.683661 21919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:34.684854 21919 hybrid_clock.cc:648] HybridClock initialized: now 1786250674684803 us; error 38 us; skew 500 ppm
I20260809 04:44:34.685173 21919 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T04:44:34Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1751 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:44:34 dist-test-slave-1jjb krb5kdc[21934](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:44:34 dist-test-slave-1jjb krb5kdc[21934](info): set up 2 sockets
Aug 09 04:44:34 dist-test-slave-1jjb krb5kdc[21934](info): commencing operation
krb5kdc: starting...
W20260809 04:44:35.825824 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:44:35 dist-test-slave-1jjb krb5kdc[21934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:44:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:44:36.074898 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:46413
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:44653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:46413
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:36.396232 21947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:36.396469 21947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:36.396623 21947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:36.402951 21947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:36.403065 21947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:36.403191 21947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:36.403290 21947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:36.403417 21947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:36.412761 21947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:44653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:46413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:46413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:36.413960 21947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:36.415117 21947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:36.426258 21956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:36.426357 21953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:36.426399 21954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:36.427163 21947 server_base.cc:1034] running on GCE node
I20260809 04:44:36.427963 21947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:36.429373 21947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:36.430640 21947 hybrid_clock.cc:648] HybridClock initialized: now 1786250676430580 us; error 43 us; skew 500 ppm
I20260809 04:44:36.430998 21947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:36 dist-test-slave-1jjb krb5kdc[21934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250676, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:36.437239 21947 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:44:36.438611 21947 webserver.cc:460] Webserver started at http://127.12.234.62:44819/ using document root <none> and password file <none>
I20260809 04:44:36.439276 21947 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:36.439412 21947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:36.439683 21947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:36.442432 21947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "88e03140ee87407f9f5b1ac88063fd19"
format_stamp: "Formatted at 2026-08-09 04:44:36 on dist-test-slave-1jjb"
server_key: "673e9d406932743ea6600f92afacb844"
server_key_iv: "7cea78e5d7ab4347d4c15e1f5aaede23"
server_key_version: "encryptionkey@0"
I20260809 04:44:36.443104 21947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "88e03140ee87407f9f5b1ac88063fd19"
format_stamp: "Formatted at 2026-08-09 04:44:36 on dist-test-slave-1jjb"
server_key: "673e9d406932743ea6600f92afacb844"
server_key_iv: "7cea78e5d7ab4347d4c15e1f5aaede23"
server_key_version: "encryptionkey@0"
I20260809 04:44:36.447708 21947 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260809 04:44:36.451030 21963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:36.452173 21947 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:44:36.452360 21947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "88e03140ee87407f9f5b1ac88063fd19"
format_stamp: "Formatted at 2026-08-09 04:44:36 on dist-test-slave-1jjb"
server_key: "673e9d406932743ea6600f92afacb844"
server_key_iv: "7cea78e5d7ab4347d4c15e1f5aaede23"
server_key_version: "encryptionkey@0"
I20260809 04:44:36.452534 21947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:36.480548 21947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:36.484306 21947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:36.484573 21947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:36.502370 21947 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:46413
I20260809 04:44:36.502389 22018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:46413 every 8 connection(s)
I20260809 04:44:36.503647 21947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:36.504549 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 21947
I20260809 04:44:36.504777 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:44:36.505380 13224 external_mini_cluster.cc:1428] Setting key 4d14b76a43185e148c4a25b88586926e
I20260809 04:44:36.507715 22019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 04:44:36 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:44:36.521966 22019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19: Bootstrap starting.
I20260809 04:44:36.525125 22019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:36.526042 22019 log.cc:824] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:36.528828 22022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:36.507262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60344 (local address 127.12.234.62:46413)
0809 04:44:36.508115 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:36.508135 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:36.508175 (+ 40us) server_negotiation.cc:407] Connection header received
0809 04:44:36.509814 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:36.509833 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:36.510196 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:36.510577 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:36.512345 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:36.514439 (+ 2094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:36.516215 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:36.517279 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:36.517489 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:36.522200 (+ 4711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:36.522208 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:36.522222 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:36.522258 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:36.524974 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:36.525654 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:36.525660 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:36.525665 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:36.525825 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:36.526201 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:36.526207 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:36.526214 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:36.526864 (+ 650us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:36.527180 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:36.527420 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 04:44:36.527816 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":326,"threads_started":1}
I20260809 04:44:36.529090 22019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19: No bootstrap required, opened a new log
I20260809 04:44:36.536196 22019 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e03140ee87407f9f5b1ac88063fd19" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46413 } }
I20260809 04:44:36.536598 22019 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:36.536734 22019 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88e03140ee87407f9f5b1ac88063fd19, State: Initialized, Role: FOLLOWER
I20260809 04:44:36.537249 22019 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e03140ee87407f9f5b1ac88063fd19" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46413 } }
I20260809 04:44:36.537485 22019 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:36.537619 22019 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:36.537791 22019 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:36.539775 22019 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e03140ee87407f9f5b1ac88063fd19" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46413 } }
I20260809 04:44:36.540398 22019 leader_election.cc:302] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88e03140ee87407f9f5b1ac88063fd19; no voters:
I20260809 04:44:36.540997 22019 leader_election.cc:288] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:36.541139 22024 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:36.541904 22024 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [term 1 LEADER]: Becoming Leader. State: Replica: 88e03140ee87407f9f5b1ac88063fd19, State: Running, Role: LEADER
I20260809 04:44:36.542438 22024 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e03140ee87407f9f5b1ac88063fd19" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46413 } }
I20260809 04:44:36.543181 22019 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:36.545225 22025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88e03140ee87407f9f5b1ac88063fd19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e03140ee87407f9f5b1ac88063fd19" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46413 } } }
I20260809 04:44:36.545619 22025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:36.546183 22026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88e03140ee87407f9f5b1ac88063fd19. Latest consensus state: current_term: 1 leader_uuid: "88e03140ee87407f9f5b1ac88063fd19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e03140ee87407f9f5b1ac88063fd19" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46413 } } }
I20260809 04:44:36.546442 22026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:36.547108 22030 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:36.552014 22030 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:36.560806 22030 catalog_manager.cc:1347] Generated new cluster ID: c4121b2d87f24accb074f377a2918140
I20260809 04:44:36.560935 22030 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:36.589732 22030 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:36.590971 22030 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:36.596267 22030 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19: Generated new TSK 0
I20260809 04:44:36.596982 22030 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
I20260809 04:44:36.655622 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:0
--local_ip_for_outbound_sockets=127.12.234.1
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:46413
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:44653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:36.968519 22046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:36.968811 22046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:36.968992 22046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:36.975718 22046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:36.975852 22046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:36.976027 22046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:44:36.985025 22046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:44653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:46413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:36.986219 22046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:36.987485 22046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:37.000406 22052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:37.000478 22053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:37.000443 22055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:37.001407 22046 server_base.cc:1034] running on GCE node
I20260809 04:44:37.002013 22046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:37.002840 22046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:37.004060 22046 hybrid_clock.cc:648] HybridClock initialized: now 1786250677004028 us; error 41 us; skew 500 ppm
I20260809 04:44:37.004411 22046 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:37 dist-test-slave-1jjb krb5kdc[21934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250677, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:37.007388 22046 init.cc:377] Logged in from keytab as kudu/127.12.234.1@KRBTEST.COM (short username kudu)
I20260809 04:44:37.008926 22046 webserver.cc:460] Webserver started at http://127.12.234.1:34593/ using document root <none> and password file <none>
I20260809 04:44:37.009554 22046 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:37.009654 22046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:37.009932 22046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:37.012588 22046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/data/instance:
uuid: "d6caedf1e34648e2b75e27bc0ea57d4b"
format_stamp: "Formatted at 2026-08-09 04:44:37 on dist-test-slave-1jjb"
server_key: "43cc6d0acd0b71cb2278fccc80113f61"
server_key_iv: "1923ab6461117a31e4a8bca1184ff91f"
server_key_version: "encryptionkey@0"
I20260809 04:44:37.013248 22046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance:
uuid: "d6caedf1e34648e2b75e27bc0ea57d4b"
format_stamp: "Formatted at 2026-08-09 04:44:37 on dist-test-slave-1jjb"
server_key: "43cc6d0acd0b71cb2278fccc80113f61"
server_key_iv: "1923ab6461117a31e4a8bca1184ff91f"
server_key_version: "encryptionkey@0"
I20260809 04:44:37.018054 22046 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 04:44:37.021256 22064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:37.022485 22046 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:44:37.022609 22046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "d6caedf1e34648e2b75e27bc0ea57d4b"
format_stamp: "Formatted at 2026-08-09 04:44:37 on dist-test-slave-1jjb"
server_key: "43cc6d0acd0b71cb2278fccc80113f61"
server_key_iv: "1923ab6461117a31e4a8bca1184ff91f"
server_key_version: "encryptionkey@0"
I20260809 04:44:37.022754 22046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:37.054067 22046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:37.057934 22046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:37.058167 22046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:37.059010 22046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:37.060226 22046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:37.060312 22046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:37.060395 22046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:37.060439 22046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:37.092216 22046 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:42133
I20260809 04:44:37.092298 22185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:42133 every 8 connection(s)
I20260809 04:44:37.093382 22046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:44:37.098775 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 22046
I20260809 04:44:37.098982 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance
I20260809 04:44:37.099620 13224 external_mini_cluster.cc:1428] Setting key 69e64720e7215be10852d6e6aa3b154b
Aug 09 04:44:37 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250677, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:44:37.112600 22189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:37.095804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:56513 (local address 127.12.234.62:46413)
0809 04:44:37.096310 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:37.096357 (+ 47us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:37.097269 (+ 912us) server_negotiation.cc:407] Connection header received
0809 04:44:37.099140 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:37.099145 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:37.099238 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:37.099377 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:37.101048 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:37.101861 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:37.103345 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:37.103753 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:37.103882 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:37.107228 (+ 3346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:37.107233 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:37.107236 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:37.107265 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:37.109876 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:37.110550 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:37.110556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:37.110564 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:37.110670 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:37.111054 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:37.111058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:37.111061 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:37.111305 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:37.111464 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:37.112038 (+ 574us) server_negotiation.cc:299] Negotiation successful
0809 04:44:37.112261 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":289,"threads_started":1}
I20260809 04:44:37.113534 22190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:37.096371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.1:56513)
0809 04:44:37.097106 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:37.097151 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:37.098834 (+ 1683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:37.099594 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:37.099601 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:37.100203 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:37.100889 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:37.100907 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:37.102023 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:37.102027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:37.103191 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:37.103202 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:37.103975 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:37.103984 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:37.104270 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:37.104917 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:37.104942 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:37.107042 (+ 2100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:37.110009 (+ 2967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:37.110017 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:37.110030 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:37.110428 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:37.110770 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:37.110775 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:37.110779 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:37.110944 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:37.111425 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:37.111432 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:37.111717 (+ 285us) client_negotiation.cc:769] Sending connection context
0809 04:44:37.111945 (+ 228us) client_negotiation.cc:240] Negotiation successful
0809 04:44:37.112434 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":314,"threads_started":1}
I20260809 04:44:37.114948 22187 heartbeater.cc:347] Connected to a master server at 127.12.234.62:46413
I20260809 04:44:37.115218 22187 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:37.115867 22187 heartbeater.cc:511] Master 127.12.234.62:46413 requested a full tablet report, sending...
I20260809 04:44:37.117832 21981 ts_manager.cc:194] Registered new tserver with Master: d6caedf1e34648e2b75e27bc0ea57d4b (127.12.234.1:42133)
I20260809 04:44:37.119249 21981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.1@KRBTEST.COM'} at 127.12.234.1:56513
WARNING: no policy specified for kudu/127.12.234.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
I20260809 04:44:37.154801 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:0
--local_ip_for_outbound_sockets=127.12.234.2
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:46413
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:44653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:37.466009 22193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:37.466236 22193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:37.466317 22193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:37.474969 22193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:37.475100 22193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:37.475282 22193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:44:37.485280 22193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:44653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:46413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:37.486505 22193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:37.487808 22193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:37.500541 22198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:37.500510 22199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:37.500692 22201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:37.501344 22193 server_base.cc:1034] running on GCE node
I20260809 04:44:37.501963 22193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:37.502846 22193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:37.504170 22193 hybrid_clock.cc:648] HybridClock initialized: now 1786250677504087 us; error 53 us; skew 500 ppm
I20260809 04:44:37.504509 22193 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:37 dist-test-slave-1jjb krb5kdc[21934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250677, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:37.507575 22193 init.cc:377] Logged in from keytab as kudu/127.12.234.2@KRBTEST.COM (short username kudu)
I20260809 04:44:37.509073 22193 webserver.cc:460] Webserver started at http://127.12.234.2:35495/ using document root <none> and password file <none>
I20260809 04:44:37.509737 22193 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:37.509883 22193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:37.510177 22193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:37.512794 22193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/data/instance:
uuid: "69a793c19d3d42c485ba598dce96d743"
format_stamp: "Formatted at 2026-08-09 04:44:37 on dist-test-slave-1jjb"
server_key: "9247062361a82bc798595a1575e34e21"
server_key_iv: "f482dcd136295cceab5f47efbdcf4c3f"
server_key_version: "encryptionkey@0"
I20260809 04:44:37.513509 22193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance:
uuid: "69a793c19d3d42c485ba598dce96d743"
format_stamp: "Formatted at 2026-08-09 04:44:37 on dist-test-slave-1jjb"
server_key: "9247062361a82bc798595a1575e34e21"
server_key_iv: "f482dcd136295cceab5f47efbdcf4c3f"
server_key_version: "encryptionkey@0"
I20260809 04:44:37.518070 22193 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:44:37.521368 22208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:37.522610 22193 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:44:37.522747 22193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "69a793c19d3d42c485ba598dce96d743"
format_stamp: "Formatted at 2026-08-09 04:44:37 on dist-test-slave-1jjb"
server_key: "9247062361a82bc798595a1575e34e21"
server_key_iv: "f482dcd136295cceab5f47efbdcf4c3f"
server_key_version: "encryptionkey@0"
I20260809 04:44:37.522914 22193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:37.555395 22193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:37.559446 22193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:37.559737 22193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:37.560663 22193 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:37.561856 22193 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:37.561941 22193 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:37.562110 22193 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:37.562157 22193 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:37.593654 22193 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:42603
I20260809 04:44:37.593691 22327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:42603 every 8 connection(s)
I20260809 04:44:37.595063 22193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:44:37.597347 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 22193
I20260809 04:44:37.597558 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance
I20260809 04:44:37.598194 13224 external_mini_cluster.cc:1428] Setting key b86d2c094b8201edb273703f5fc9640b
Aug 09 04:44:37 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250677, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:44:37.615569 22189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:37.597861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:36567 (local address 127.12.234.62:46413)
0809 04:44:37.598073 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:37.598077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:37.599561 (+ 1484us) server_negotiation.cc:407] Connection header received
0809 04:44:37.601607 (+ 2046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:37.601612 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:37.601700 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:37.601807 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:37.603221 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:37.603899 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:37.605438 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:37.605861 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:37.605999 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:37.609270 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:37.609275 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:37.609278 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:37.609308 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:37.612298 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:37.613009 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:37.613016 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:37.613022 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:37.613171 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:37.613573 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:37.613580 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:37.613586 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:37.613948 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:37.614107 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:37.614876 (+ 769us) server_negotiation.cc:299] Negotiation successful
0809 04:44:37.615122 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 04:44:37.616194 22330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:37.598529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.2:36567)
0809 04:44:37.599352 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:37.599428 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:37.601344 (+ 1916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:37.602036 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:37.602043 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:37.602512 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:37.603058 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:37.603075 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:37.604115 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:37.604121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:37.605286 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:37.605298 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:37.606102 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:37.606109 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:37.606321 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:37.606961 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:37.606984 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:37.609038 (+ 2054us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:37.612429 (+ 3391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:37.612435 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:37.612463 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:37.612875 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:37.613274 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:37.613277 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:37.613279 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:37.613456 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:37.614069 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:37.614074 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:37.614482 (+ 408us) client_negotiation.cc:769] Sending connection context
0809 04:44:37.614735 (+ 253us) client_negotiation.cc:240] Negotiation successful
0809 04:44:37.615088 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":364,"threads_started":1}
I20260809 04:44:37.617623 22328 heartbeater.cc:347] Connected to a master server at 127.12.234.62:46413
I20260809 04:44:37.617995 22328 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:37.618816 22328 heartbeater.cc:511] Master 127.12.234.62:46413 requested a full tablet report, sending...
I20260809 04:44:37.620528 21981 ts_manager.cc:194] Registered new tserver with Master: 69a793c19d3d42c485ba598dce96d743 (127.12.234.2:42603)
I20260809 04:44:37.621320 21981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.2@KRBTEST.COM'} at 127.12.234.2:36567
WARNING: no policy specified for kudu/127.12.234.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
I20260809 04:44:37.657238 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:0
--local_ip_for_outbound_sockets=127.12.234.3
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:46413
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:44653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:37.968222 22334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:37.968461 22334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:37.968613 22334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:37.975152 22334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:37.975262 22334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:37.975397 22334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:44:37.984534 22334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:44653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:46413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:37.985666 22334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:37.986933 22334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:37.999614 22339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:37.999676 22340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:37.999722 22342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:38.000718 22334 server_base.cc:1034] running on GCE node
I20260809 04:44:38.001235 22334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:38.002018 22334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:38.003216 22334 hybrid_clock.cc:648] HybridClock initialized: now 1786250678003187 us; error 28 us; skew 500 ppm
I20260809 04:44:38.003517 22334 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:38 dist-test-slave-1jjb krb5kdc[21934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250678, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:38.006678 22334 init.cc:377] Logged in from keytab as kudu/127.12.234.3@KRBTEST.COM (short username kudu)
I20260809 04:44:38.008145 22334 webserver.cc:460] Webserver started at http://127.12.234.3:41471/ using document root <none> and password file <none>
I20260809 04:44:38.008816 22334 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:38.008946 22334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:38.009276 22334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:38.012140 22334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/data/instance:
uuid: "72616aee6f5847bba09fa49aed09689c"
format_stamp: "Formatted at 2026-08-09 04:44:38 on dist-test-slave-1jjb"
server_key: "f8005540c61bb9207d08fad4b761aeed"
server_key_iv: "56d5350ec09174d8a25bc9f1df63561d"
server_key_version: "encryptionkey@0"
I20260809 04:44:38.012964 22334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance:
uuid: "72616aee6f5847bba09fa49aed09689c"
format_stamp: "Formatted at 2026-08-09 04:44:38 on dist-test-slave-1jjb"
server_key: "f8005540c61bb9207d08fad4b761aeed"
server_key_iv: "56d5350ec09174d8a25bc9f1df63561d"
server_key_version: "encryptionkey@0"
I20260809 04:44:38.017777 22334 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 04:44:38.021123 22349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:38.022331 22334 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:44:38.022492 22334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "72616aee6f5847bba09fa49aed09689c"
format_stamp: "Formatted at 2026-08-09 04:44:38 on dist-test-slave-1jjb"
server_key: "f8005540c61bb9207d08fad4b761aeed"
server_key_iv: "56d5350ec09174d8a25bc9f1df63561d"
server_key_version: "encryptionkey@0"
I20260809 04:44:38.022670 22334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:38.046583 22334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:38.050968 22334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:38.051273 22334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:38.052254 22334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:38.053428 22334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:38.053496 22334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:38.053653 22334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:38.053702 22334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:38.087013 22334 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:39357
I20260809 04:44:38.087133 22473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:39357 every 8 connection(s)
I20260809 04:44:38.088339 22334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:44:38.089896 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 22334
I20260809 04:44:38.090075 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance
I20260809 04:44:38.090600 13224 external_mini_cluster.cc:1428] Setting key d22a7f6aec31930a5722d0fe9d4b84c7
Aug 09 04:44:38 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250678, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:44:38.112131 22189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:38.091014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:55543 (local address 127.12.234.62:46413)
0809 04:44:38.091215 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:38.091221 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:38.096779 (+ 5558us) server_negotiation.cc:407] Connection header received
0809 04:44:38.098716 (+ 1937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:38.098721 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:38.098785 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:38.098872 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:38.100363 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:38.101070 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:38.102659 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:38.103035 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:38.103131 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:38.106837 (+ 3706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:38.106843 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:38.106846 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:38.106879 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:38.109549 (+ 2670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:38.110218 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:38.110222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:38.110224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:38.110381 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:38.110740 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:38.110743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:38.110745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:38.111002 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:38.111115 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:38.111636 (+ 521us) server_negotiation.cc:299] Negotiation successful
0809 04:44:38.111817 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 04:44:38.112993 22476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:38.091609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.3:55543)
0809 04:44:38.096592 (+ 4983us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:38.096654 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:38.098424 (+ 1770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:38.099077 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:38.099084 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:38.099588 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:38.100211 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:38.100231 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:38.101204 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:38.101207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:38.102418 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:38.102430 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:38.103231 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:38.103236 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:38.103438 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:38.104149 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:38.104180 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:38.106635 (+ 2455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:38.109706 (+ 3071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:38.109712 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:38.109722 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:38.110107 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:38.110470 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:44:38.110472 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:44:38.110474 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:44:38.110629 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:44:38.111102 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:38.111107 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:38.111349 (+ 242us) client_negotiation.cc:769] Sending connection context
0809 04:44:38.111546 (+ 197us) client_negotiation.cc:240] Negotiation successful
0809 04:44:38.111871 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4756,"thread_start_us":4527,"threads_started":1}
I20260809 04:44:38.114300 22474 heartbeater.cc:347] Connected to a master server at 127.12.234.62:46413
I20260809 04:44:38.114547 22474 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:38.115195 22474 heartbeater.cc:511] Master 127.12.234.62:46413 requested a full tablet report, sending...
I20260809 04:44:38.116592 21981 ts_manager.cc:194] Registered new tserver with Master: 72616aee6f5847bba09fa49aed09689c (127.12.234.3:39357)
I20260809 04:44:38.117601 21981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.3@KRBTEST.COM'} at 127.12.234.3:55543
I20260809 04:44:38.118333 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:44:38.121783 22187 heartbeater.cc:503] Master 127.12.234.62:46413 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.12.234.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.234.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.234.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.1.keytab.
Entry for principal oryx/127.12.234.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.1.keytab.
WARNING: no policy specified for oryx/127.12.234.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.234.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.234.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.2.keytab.
Entry for principal oryx/127.12.234.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.2.keytab.
WARNING: no policy specified for oryx/127.12.234.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.234.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.234.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.3.keytab.
Entry for principal oryx/127.12.234.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.3.keytab.
I20260809 04:44:38.259356 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 22046
Aug 09 04:44:38 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.1@KRBTEST.COM
I20260809 04:44:38.274535 22485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:38.260440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33876 (local address 127.12.234.1:42133)
0809 04:44:38.261101 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:38.261109 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:38.261136 (+ 27us) server_negotiation.cc:407] Connection header received
0809 04:44:38.261200 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:38.261206 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:38.261433 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:38.261568 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:38.262851 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:38.263824 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:38.265092 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:38.265620 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:38.265713 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:38.269439 (+ 3726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:38.269450 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:38.269464 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:38.269495 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:38.272207 (+ 2712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:38.272781 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:38.272789 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:38.272791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:38.272933 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:38.273242 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:38.273246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:38.273247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:38.273603 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:38.273836 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:38.274009 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 04:44:38.274203 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":358,"threads_started":1}
I20260809 04:44:38.275794 22181 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:38.356858 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 22046
I20260809 04:44:38.375743 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 22193
Aug 09 04:44:38 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.2@KRBTEST.COM
I20260809 04:44:38.391206 22487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:38.376617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48784 (local address 127.12.234.2:42603)
0809 04:44:38.377171 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:38.377177 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:38.377203 (+ 26us) server_negotiation.cc:407] Connection header received
0809 04:44:38.377261 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:38.377267 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:38.377452 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:38.377593 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:38.378711 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:38.379662 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:38.380946 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:38.381458 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:38.381558 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:38.385899 (+ 4341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:38.385911 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:38.385924 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:38.385959 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:38.388789 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:38.389427 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:38.389434 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:38.389436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:38.389567 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:38.389880 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:38.389883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:38.389884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:38.390209 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:38.390424 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:38.390743 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 04:44:38.390920 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":277,"threads_started":1}
I20260809 04:44:38.392367 22322 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:38.470064 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 22193
I20260809 04:44:38.488179 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 22334
Aug 09 04:44:38 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.3@KRBTEST.COM
I20260809 04:44:38.504211 22489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:38.489162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39682 (local address 127.12.234.3:39357)
0809 04:44:38.489779 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:38.489786 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:38.489815 (+ 29us) server_negotiation.cc:407] Connection header received
0809 04:44:38.489877 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:38.489883 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:38.490082 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:38.490264 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:38.491674 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:38.492835 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:38.494050 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:38.494596 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:38.494805 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:38.498988 (+ 4183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:38.498999 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:38.499011 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:38.499045 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:38.501586 (+ 2541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:38.502198 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:38.502204 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:38.502206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:38.502323 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:38.502685 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:38.502690 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:38.502693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:38.503034 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:38.503339 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:38.503600 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 04:44:38.503845 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":323,"threads_started":1}
I20260809 04:44:38.505378 22469 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:38.582222 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 22334
I20260809 04:44:38.601135 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 21947
I20260809 04:44:38.602412 22014 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:38.699501 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 21947
I20260809 04:44:38.722119 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:46413
--webserver_interface=127.12.234.62
--webserver_port=44819
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:44653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:46413
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:44:39.034214 22493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:39.034525 22493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:39.034672 22493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:39.041440 22493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:39.041522 22493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:39.041563 22493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:39.041610 22493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:39.041643 22493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:39.050411 22493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:44653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:46413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:46413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=44819
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:39.051487 22493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:39.052736 22493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:39.063987 22499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:39.064055 22503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:39.064181 22500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:39.065021 22493 server_base.cc:1034] running on GCE node
I20260809 04:44:39.065785 22493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:39.066939 22493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:39.068182 22493 hybrid_clock.cc:648] HybridClock initialized: now 1786250679068141 us; error 24 us; skew 500 ppm
I20260809 04:44:39.068526 22493 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:39 dist-test-slave-1jjb krb5kdc[21934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250679, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:39.071425 22493 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:44:39.072861 22493 webserver.cc:460] Webserver started at http://127.12.234.62:44819/ using document root <none> and password file <none>
I20260809 04:44:39.073503 22493 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:39.073654 22493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:39.078636 22493 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:44:39.081185 22510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:39.082406 22493 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:44:39.082588 22493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "88e03140ee87407f9f5b1ac88063fd19"
format_stamp: "Formatted at 2026-08-09 04:44:36 on dist-test-slave-1jjb"
server_key: "673e9d406932743ea6600f92afacb844"
server_key_iv: "7cea78e5d7ab4347d4c15e1f5aaede23"
server_key_version: "encryptionkey@0"
I20260809 04:44:39.083491 22493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:39.131237 22493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:39.135069 22493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:39.135332 22493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:39.153625 22493 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:46413
I20260809 04:44:39.153657 22568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:46413 every 8 connection(s)
I20260809 04:44:39.154918 22493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:39.156919 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 22493
I20260809 04:44:39.157807 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:42133
--local_ip_for_outbound_sockets=127.12.234.1
--tserver_master_addrs=127.12.234.62:46413
--webserver_port=34593
--webserver_interface=127.12.234.1
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:44653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.12.234.1
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:44:39.163342 22569 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:44:39.168149 22569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19: Bootstrap starting.
I20260809 04:44:39.186456 22569 log.cc:824] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:39.195036 22569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:44:39.195551 22569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19: Bootstrap complete.
I20260809 04:44:39.204319 22569 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e03140ee87407f9f5b1ac88063fd19" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46413 } }
I20260809 04:44:39.205135 22569 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88e03140ee87407f9f5b1ac88063fd19, State: Initialized, Role: FOLLOWER
I20260809 04:44:39.205765 22569 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e03140ee87407f9f5b1ac88063fd19" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46413 } }
I20260809 04:44:39.206048 22569 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:39.206175 22569 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:39.206365 22569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [term 1 FOLLOWER]: Advancing to term 2
I20260809 04:44:39.210090 22569 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e03140ee87407f9f5b1ac88063fd19" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46413 } }
I20260809 04:44:39.210561 22569 leader_election.cc:302] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88e03140ee87407f9f5b1ac88063fd19; no voters:
I20260809 04:44:39.211186 22569 leader_election.cc:288] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 04:44:39.211318 22573 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 04:44:39.212597 22573 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [term 2 LEADER]: Becoming Leader. State: Replica: 88e03140ee87407f9f5b1ac88063fd19, State: Running, Role: LEADER
I20260809 04:44:39.213202 22573 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e03140ee87407f9f5b1ac88063fd19" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46413 } }
I20260809 04:44:39.213424 22569 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:39.216141 22574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "88e03140ee87407f9f5b1ac88063fd19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e03140ee87407f9f5b1ac88063fd19" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46413 } } }
I20260809 04:44:39.216495 22574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:39.215968 22575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88e03140ee87407f9f5b1ac88063fd19. Latest consensus state: current_term: 2 leader_uuid: "88e03140ee87407f9f5b1ac88063fd19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88e03140ee87407f9f5b1ac88063fd19" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46413 } } }
I20260809 04:44:39.217079 22575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:39.229091 22581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:39.234491 22581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:39.235919 22581 catalog_manager.cc:1259] Loaded cluster ID: c4121b2d87f24accb074f377a2918140
I20260809 04:44:39.236047 22581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:39.239679 22581 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:39.241026 22581 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 88e03140ee87407f9f5b1ac88063fd19: Loaded TSK: 0
I20260809 04:44:39.242367 22581 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 04:44:39.492520 22571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:39.492825 22571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:39.493003 22571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:39.499810 22571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:39.499964 22571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:39.500176 22571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:44:39.509093 22571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:44653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.1.keytab
--principal=oryx/127.12.234.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:42133
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=34593
--tserver_master_addrs=127.12.234.62:46413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:39.510308 22571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:39.511552 22571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:39.527549 22596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:39.527742 22597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:39.527595 22599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:39.528512 22571 server_base.cc:1034] running on GCE node
I20260809 04:44:39.529143 22571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:39.530026 22571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:39.531242 22571 hybrid_clock.cc:648] HybridClock initialized: now 1786250679531196 us; error 42 us; skew 500 ppm
I20260809 04:44:39.531522 22571 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:39 dist-test-slave-1jjb krb5kdc[21934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250679, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:39.534629 22571 init.cc:377] Logged in from keytab as oryx/127.12.234.1@KRBTEST.COM (short username oryx)
I20260809 04:44:39.536031 22571 webserver.cc:460] Webserver started at http://127.12.234.1:34593/ using document root <none> and password file <none>
I20260809 04:44:39.536655 22571 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:39.536798 22571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:39.541605 22571 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260809 04:44:39.543998 22607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:39.545279 22571 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:44:39.545426 22571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "d6caedf1e34648e2b75e27bc0ea57d4b"
format_stamp: "Formatted at 2026-08-09 04:44:37 on dist-test-slave-1jjb"
server_key: "43cc6d0acd0b71cb2278fccc80113f61"
server_key_iv: "1923ab6461117a31e4a8bca1184ff91f"
server_key_version: "encryptionkey@0"
I20260809 04:44:39.546268 22571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:39.593755 22571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:39.597841 22571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:39.598102 22571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:39.599097 22571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:39.600433 22571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:39.600549 22571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:39.600688 22571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:39.600769 22571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:39.633625 22571 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:42133
I20260809 04:44:39.633701 22730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:42133 every 8 connection(s)
I20260809 04:44:39.634989 22571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:44:39.636511 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 22571
I20260809 04:44:39.637099 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:42603
--local_ip_for_outbound_sockets=127.12.234.2
--tserver_master_addrs=127.12.234.62:46413
--webserver_port=35495
--webserver_interface=127.12.234.2
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:44653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.12.234.2
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:44:39 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:39 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 04:44:39.966856 22735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:39.967125 22735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:39.967278 22735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:39.973973 22735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:39.974098 22735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:39.974264 22735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:44:39.983105 22735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:44653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.2.keytab
--principal=oryx/127.12.234.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:42603
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=35495
--tserver_master_addrs=127.12.234.62:46413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:39.984372 22735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:39.985627 22735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:40.002317 22743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:40.002663 22742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:40.002786 22745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:40.003413 22735 server_base.cc:1034] running on GCE node
I20260809 04:44:40.004411 22735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:40.005862 22735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:40.007189 22735 hybrid_clock.cc:648] HybridClock initialized: now 1786250680007101 us; error 45 us; skew 500 ppm
I20260809 04:44:40.007712 22735 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:40 dist-test-slave-1jjb krb5kdc[21934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250680, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:40.012975 22735 init.cc:377] Logged in from keytab as oryx/127.12.234.2@KRBTEST.COM (short username oryx)
I20260809 04:44:40.015090 22735 webserver.cc:460] Webserver started at http://127.12.234.2:35495/ using document root <none> and password file <none>
I20260809 04:44:40.016179 22735 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:40.016381 22735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:40.025682 22735 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.007s sys 0.001s
I20260809 04:44:40.029954 22752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:40.031898 22735 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.004s
I20260809 04:44:40.032171 22735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "69a793c19d3d42c485ba598dce96d743"
format_stamp: "Formatted at 2026-08-09 04:44:37 on dist-test-slave-1jjb"
server_key: "9247062361a82bc798595a1575e34e21"
server_key_iv: "f482dcd136295cceab5f47efbdcf4c3f"
server_key_version: "encryptionkey@0"
I20260809 04:44:40.034010 22735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:40.083007 22735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:40.090185 22735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:40.090582 22735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:40.092128 22735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:40.094405 22735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:40.094559 22735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:40.094698 22735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:40.094797 22735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:40.133728 22735 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:42603
I20260809 04:44:40.133737 22873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:42603 every 8 connection(s)
I20260809 04:44:40.136348 22735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:44:40.144030 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 22735
I20260809 04:44:40.144675 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:39357
--local_ip_for_outbound_sockets=127.12.234.3
--tserver_master_addrs=127.12.234.62:46413
--webserver_port=41471
--webserver_interface=127.12.234.3
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:44653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.12.234.3
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:44:40 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:40 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 04:44:40.783411 22878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:40.783915 22878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:40.784173 22878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:40.796701 22733 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.137s user 0.005s sys 0.013s
W20260809 04:44:40.798735 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:39.637785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:49767 (local address 127.12.234.62:46413)
0809 04:44:39.649801 (+ 12016us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:39.649815 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:39.649872 (+ 57us) server_negotiation.cc:407] Connection header received
0809 04:44:39.650319 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:39.650339 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:39.650812 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:39.653150 (+ 2338us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:39.653504 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:39.655209 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:39.657135 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:39.657806 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:39.657938 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:40.797983 (+1140045us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.1:49767: BlockingRecv error: recv got EOF from 127.12.234.1:49767 (error 108)
Metrics: {"server-negotiator.queue_time_us":11779,"thread_start_us":11570,"threads_started":1}
I20260809 04:44:40.798750 22733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:39.637805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.1:49767)
0809 04:44:39.644700 (+ 6895us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:39.644762 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:39.647320 (+ 2558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:39.651455 (+ 4135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:39.651464 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:39.652225 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:39.652914 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:39.652933 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:39.655380 (+ 2447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:39.655385 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:39.656976 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:39.656988 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:39.658073 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:39.658088 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:39.658481 (+ 393us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:39.659376 (+ 895us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:39.659414 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:40.797564 (+1138150us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":6588,"thread_start_us":6389,"threads_started":1}
W20260809 04:44:40.800292 22731 heartbeater.cc:644] Failed to heartbeat to 127.12.234.62:46413 (0 consecutive failures): Not authorized: Failed to ping master at 127.12.234.62:46413: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
W20260809 04:44:40.802225 22878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:40.802413 22878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:40.802695 22878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
Aug 09 04:44:40 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:40 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:40.809477 22733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:40.801012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.1:35487)
0809 04:44:40.801264 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:40.801282 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:40.801423 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:40.801760 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:40.801764 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:40.802149 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:40.802300 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:40.802335 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:40.803398 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:40.803403 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:40.804364 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:40.804375 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:40.805011 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:40.805017 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:40.805192 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:40.805768 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:40.805783 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:40.809178 (+ 3395us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138}
W20260809 04:44:40.809607 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:40.801118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:35487 (local address 127.12.234.62:46413)
0809 04:44:40.801255 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:40.801260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:40.801372 (+ 112us) server_negotiation.cc:407] Connection header received
0809 04:44:40.801555 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:40.801561 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:40.801627 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:40.801726 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:40.802473 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:40.803237 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:40.804499 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:40.804847 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:40.804970 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:40.809364 (+ 4394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.1:35487: BlockingRecv error: recv got EOF from 127.12.234.1:35487 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 04:44:40 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:40 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:40.818090 22733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:40.810352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.1:34631)
0809 04:44:40.810609 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:40.810626 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:40.810749 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:40.811091 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:40.811095 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:40.811428 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:40.811574 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:40.811581 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:40.812510 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:40.812515 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:40.813356 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:40.813367 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:40.814158 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:40.814164 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:40.814327 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:40.814943 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:40.814958 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:40.817858 (+ 2900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161}
W20260809 04:44:40.818243 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:40.810458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:34631 (local address 127.12.234.62:46413)
0809 04:44:40.810591 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:40.810596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:40.810700 (+ 104us) server_negotiation.cc:407] Connection header received
0809 04:44:40.810881 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:40.810886 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:40.810963 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:40.811063 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:40.811694 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:40.812385 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:40.813485 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:40.814022 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:40.814145 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:40.818019 (+ 3874us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.1:34631: BlockingRecv error: recv got EOF from 127.12.234.1:34631 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260809 04:44:40.818662 22731 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 04:44:40.825469 22878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:44653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/krb5kdc/oryx_127.12.234.3.keytab
--principal=oryx/127.12.234.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:39357
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=41471
--tserver_master_addrs=127.12.234.62:46413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:40.827548 22878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:40.829870 22878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:40.848626 22886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:40.848727 22885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:40.849825 22888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:40.850643 22878 server_base.cc:1034] running on GCE node
I20260809 04:44:40.851563 22878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:40.853029 22878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:40.854307 22878 hybrid_clock.cc:648] HybridClock initialized: now 1786250680854230 us; error 47 us; skew 500 ppm
I20260809 04:44:40.854784 22878 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:44:40 dist-test-slave-1jjb krb5kdc[21934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250680, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.234.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:44:40.859934 22878 init.cc:377] Logged in from keytab as oryx/127.12.234.3@KRBTEST.COM (short username oryx)
I20260809 04:44:40.862053 22878 webserver.cc:460] Webserver started at http://127.12.234.3:41471/ using document root <none> and password file <none>
I20260809 04:44:40.863171 22878 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:40.863369 22878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:40.872268 22878 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.007s sys 0.000s
I20260809 04:44:40.876513 22896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:40.878434 22878 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260809 04:44:40.878667 22878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "72616aee6f5847bba09fa49aed09689c"
format_stamp: "Formatted at 2026-08-09 04:44:38 on dist-test-slave-1jjb"
server_key: "f8005540c61bb9207d08fad4b761aeed"
server_key_iv: "56d5350ec09174d8a25bc9f1df63561d"
server_key_version: "encryptionkey@0"
I20260809 04:44:40.880525 22878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:40.910759 22878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:40.917482 22878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:40.917829 22878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:40.919379 22878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:40.921602 22878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:40.921762 22878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:40.921891 22878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:40.921996 22878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:40.964380 23020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:39357 every 8 connection(s)
I20260809 04:44:40.964401 22878 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:39357
I20260809 04:44:40.966717 22878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:44:40.973600 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 22878
Aug 09 04:44:40 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:40 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:41.002754 23025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:40.974631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60348 (local address 127.12.234.62:46413)
0809 04:44:40.983640 (+ 9009us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:40.983646 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:40.983664 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:44:40.983712 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:40.983717 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:40.983781 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:40.983892 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:40.986864 (+ 2972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:40.988219 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:40.990872 (+ 2653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:40.991571 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:40.991698 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:40.996183 (+ 4485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:40.996195 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:40.996217 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:40.996269 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:40.999728 (+ 3459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:41.000457 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:41.000464 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:41.000469 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:41.000622 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:44:41.000990 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:44:41.000994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:44:41.000995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:44:41.001412 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:41.001714 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:41.002110 (+ 396us) server_negotiation.cc:299] Negotiation successful
0809 04:44:41.002474 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8016,"thread_start_us":8062,"threads_started":1}
W20260809 04:44:41.390560 22877 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.235s user 0.005s sys 0.014s
W20260809 04:44:41.392194 22876 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:40.139639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:36055 (local address 127.12.234.62:46413)
0809 04:44:40.140257 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:40.140266 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:40.141397 (+ 1131us) server_negotiation.cc:407] Connection header received
0809 04:44:40.145241 (+ 3844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:40.145251 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:40.145366 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:40.145513 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:40.147613 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:40.149323 (+ 1710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:40.151980 (+ 2657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:40.152911 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:40.153101 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:41.391882 (+1238781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.2:36055: BlockingRecv error: recv got EOF from 127.12.234.2:36055 (error 108)
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":312,"threads_started":1}
I20260809 04:44:41.393400 22877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:40.140081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.2:36055)
0809 04:44:40.141174 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:40.141236 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:40.144659 (+ 3423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:40.145784 (+ 1125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:40.145799 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:40.146590 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:40.147392 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:40.147421 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:40.149537 (+ 2116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:40.149545 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:40.151795 (+ 2250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:40.151825 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:40.153531 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:40.153541 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:40.153983 (+ 442us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:40.154939 (+ 956us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:40.154984 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:41.391759 (+1236775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":389,"threads_started":1}
W20260809 04:44:41.394413 22874 heartbeater.cc:644] Failed to heartbeat to 127.12.234.62:46413 (0 consecutive failures): Not authorized: Failed to ping master at 127.12.234.62:46413: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Aug 09 04:44:41 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:41 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:41.408161 22877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:41.395103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.2:44717)
0809 04:44:41.395367 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:41.395394 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:41.395564 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:41.395905 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:41.395912 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:41.396534 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:41.396870 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:41.396890 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:41.397858 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:41.397865 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:41.400113 (+ 2248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:41.400134 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:41.400908 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:41.400917 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:41.401165 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:41.402042 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:41.402073 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:41.407846 (+ 5773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":77}
W20260809 04:44:41.408346 22876 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:41.395201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:44717 (local address 127.12.234.62:46413)
0809 04:44:41.395359 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:41.395364 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:41.395499 (+ 135us) server_negotiation.cc:407] Connection header received
0809 04:44:41.395685 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:41.395689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:41.395750 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:41.395833 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:41.397036 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:41.397702 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:41.400296 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:41.400676 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:41.400768 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:41.408032 (+ 7264us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.2:44717: BlockingRecv error: recv got EOF from 127.12.234.2:44717 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 04:44:41 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:41 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 04:44:41.417796 22876 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:41.409327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:48025 (local address 127.12.234.62:46413)
0809 04:44:41.409488 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:41.409493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:41.409509 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:44:41.409649 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:41.409655 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:41.409732 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:41.409833 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:41.410499 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:41.411188 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:41.412406 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:41.412794 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:41.412919 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:41.417606 (+ 4687us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.2:48025: BlockingRecv error: recv got EOF from 127.12.234.2:48025 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:44:41.417847 22877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:41.409217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.2:48025)
0809 04:44:41.409402 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:41.409421 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:41.409519 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:41.409869 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:41.409874 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:41.410189 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:41.410354 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:41.410362 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:41.411322 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:41.411326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:41.412264 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:41.412276 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:41.412934 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:41.412939 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:41.413117 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:41.413768 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:41.413784 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:41.417472 (+ 3688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":66}
W20260809 04:44:41.418159 22874 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 04:44:41 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:41 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:41.833819 23028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:41.819399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.1:39089)
0809 04:44:41.820127 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:41.820146 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:41.820320 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:41.820763 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:41.820768 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:41.821257 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:41.821400 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:41.821408 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:41.822328 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:41.822334 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:41.823262 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:41.823273 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:41.823949 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:41.823954 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:41.824188 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:41.829720 (+ 5532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:41.829738 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:41.833557 (+ 3819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":672,"threads_started":1}
W20260809 04:44:41.834292 22876 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:41.819442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:39089 (local address 127.12.234.62:46413)
0809 04:44:41.819664 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:41.819673 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:41.820219 (+ 546us) server_negotiation.cc:407] Connection header received
0809 04:44:41.820449 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:41.820455 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:41.820601 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:41.820703 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:41.821553 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:41.822194 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:41.823400 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:41.823780 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:41.823905 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:41.833955 (+ 10050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.1:39089: BlockingRecv error: recv got EOF from 127.12.234.1:39089 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260809 04:44:42.063776 23024 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.081s user 0.007s sys 0.007s
W20260809 04:44:42.065174 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:40.969708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:60053 (local address 127.12.234.62:46413)
0809 04:44:40.969890 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:40.969894 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:40.971623 (+ 1729us) server_negotiation.cc:407] Connection header received
0809 04:44:40.973417 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:40.973427 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:40.973558 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:40.973703 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:40.975817 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:40.977121 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:40.980226 (+ 3105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:40.981107 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:40.981219 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:42.064875 (+1083656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.3:60053: BlockingRecv error: recv got EOF from 127.12.234.3:60053 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 04:44:42.065618 23024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:40.970618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.3:60053)
0809 04:44:40.971468 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:40.971507 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:40.973085 (+ 1578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:40.973965 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:40.973977 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:40.974689 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:40.975551 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:40.975579 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:40.977356 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:40.977363 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:40.979988 (+ 2625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:40.980009 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:40.981639 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:40.981648 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:40.982037 (+ 389us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:40.982807 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:40.982853 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:42.064650 (+1081797us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":379,"threads_started":1}
W20260809 04:44:42.066563 23021 heartbeater.cc:644] Failed to heartbeat to 127.12.234.62:46413 (0 consecutive failures): Not authorized: Failed to ping master at 127.12.234.62:46413: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Aug 09 04:44:42 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:42 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:42.076047 23024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:42.067169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.3:39421)
0809 04:44:42.067381 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:42.067402 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:42.067521 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:42.067958 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:42.067963 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:42.068492 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:42.068711 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:42.068725 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:42.069986 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:42.069990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:42.070948 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:42.070959 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:42.071734 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:42.071739 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:42.071852 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:42.072580 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:42.072596 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:42.075872 (+ 3276us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":61}
W20260809 04:44:42.076385 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:42.067241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:39421 (local address 127.12.234.62:46413)
0809 04:44:42.067474 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:42.067479 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:42.067497 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:44:42.067660 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:42.067666 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:42.067765 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:42.067876 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:42.068881 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:42.069855 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:42.071081 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:42.071546 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:42.071687 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:42.076054 (+ 4367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.3:39421: BlockingRecv error: recv got EOF from 127.12.234.3:39421 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
Aug 09 04:44:42 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:42 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:42.084626 23024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:42.076902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.3:51769)
0809 04:44:42.077085 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:42.077102 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:42.077201 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:42.077453 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:42.077468 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:42.077695 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:42.077821 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:42.077827 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:42.078706 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:42.078710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:42.079719 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:42.079734 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:42.080717 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:42.080724 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:42.080910 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:42.081521 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:42.081542 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:42.084461 (+ 2919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":61}
W20260809 04:44:42.084925 23021 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 04:44:42.085203 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:42.077085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:51769 (local address 127.12.234.62:46413)
0809 04:44:42.077226 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:42.077230 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:42.077247 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:44:42.077285 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:42.077288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:42.077339 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:42.077417 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:42.077929 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:42.078588 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:42.079872 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:42.080552 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:42.080663 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:42.084641 (+ 3978us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.3:51769: BlockingRecv error: recv got EOF from 127.12.234.3:51769 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 04:44:42 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:42 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:42.426829 23029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:42.418747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.2:41303)
0809 04:44:42.419300 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:42.419315 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:42.419493 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:42.419834 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:42.419838 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:42.420330 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:42.420484 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:42.420492 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:42.421437 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:42.421442 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:42.422296 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:42.422306 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:42.422972 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:42.422978 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:42.423125 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:42.423642 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:42.423659 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:42.426628 (+ 2969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":335,"threads_started":1}
W20260809 04:44:42.427222 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:42.418821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:41303 (local address 127.12.234.62:46413)
0809 04:44:42.419056 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:42.419064 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:42.419418 (+ 354us) server_negotiation.cc:407] Connection header received
0809 04:44:42.419647 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:42.419650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:42.419709 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:42.419801 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:42.420614 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:42.421308 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:42.422429 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:42.422795 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:42.422936 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:42.426805 (+ 3869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.2:41303: BlockingRecv error: recv got EOF from 127.12.234.2:41303 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 04:44:42 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:42 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:42.844247 23030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:42.835344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.1:47153)
0809 04:44:42.835865 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:42.835881 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:42.835987 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:42.836294 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:42.836297 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:42.836702 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:42.836874 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:42.836881 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:42.837837 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:42.837840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:42.838722 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:42.838733 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:42.839502 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:42.839506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:42.839650 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:42.840270 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:42.840286 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:42.843854 (+ 3568us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":317,"threads_started":1}
W20260809 04:44:42.844336 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:42.835460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:47153 (local address 127.12.234.62:46413)
0809 04:44:42.835625 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:42.835629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:42.835957 (+ 328us) server_negotiation.cc:407] Connection header received
0809 04:44:42.836102 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:42.836105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:42.836170 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:42.836253 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:42.837008 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:42.837720 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:42.838862 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:42.839363 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:42.839474 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:42.844030 (+ 4556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.1:47153: BlockingRecv error: recv got EOF from 127.12.234.1:47153 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 04:44:43 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:43 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:43.094911 23031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:43.085551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.3:53631)
0809 04:44:43.086138 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:43.086158 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:43.086279 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:43.086608 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:43.086612 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:43.087080 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:43.087311 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:43.087321 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:43.088689 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:43.088697 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:43.089702 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:43.089714 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:43.090427 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:43.090433 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:43.090609 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:43.091145 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:43.091163 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:43.094571 (+ 3408us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":329,"threads_started":1}
W20260809 04:44:43.095005 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:43.085645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:53631 (local address 127.12.234.62:46413)
0809 04:44:43.085800 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:43.085804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:43.086233 (+ 429us) server_negotiation.cc:407] Connection header received
0809 04:44:43.086380 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:43.086384 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:43.086443 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:43.086552 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:43.087429 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:43.088556 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:43.089850 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:43.090243 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:43.090375 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:43.094776 (+ 4401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.3:53631: BlockingRecv error: recv got EOF from 127.12.234.3:53631 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 09 04:44:43 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:43 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:43.436440 23032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:43.427899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.2:44567)
0809 04:44:43.428418 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:43.428440 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:43.428688 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:43.429230 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:43.429234 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:43.429706 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:43.429907 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:43.429917 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:43.430824 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:43.430828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:43.431669 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:43.431680 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:43.432329 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:43.432334 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:43.432457 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:43.432996 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:43.433014 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:43.436212 (+ 3198us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":303,"threads_started":1}
W20260809 04:44:43.436596 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:43.427934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:44567 (local address 127.12.234.62:46413)
0809 04:44:43.428172 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:43.428178 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:43.428601 (+ 423us) server_negotiation.cc:407] Connection header received
0809 04:44:43.428916 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:43.428920 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:43.429018 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:43.429138 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:43.430038 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:43.430698 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:43.431784 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:43.432189 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:43.432269 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:43.436384 (+ 4115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.2:44567: BlockingRecv error: recv got EOF from 127.12.234.2:44567 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 04:44:43 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:43 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:43.856482 23033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:43.845348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.1:42379)
0809 04:44:43.845854 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:43.845870 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:43.846035 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:43.846372 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:43.846378 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:43.846842 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:43.847045 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:43.847056 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:43.848755 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:43.848763 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:43.850573 (+ 1810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:43.850592 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:43.851661 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:43.851666 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:43.851817 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:43.852423 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:43.852447 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:43.856133 (+ 3686us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":252,"threads_started":1}
W20260809 04:44:43.856575 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:43.845503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:42379 (local address 127.12.234.62:46413)
0809 04:44:43.845697 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:43.845702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:43.845981 (+ 279us) server_negotiation.cc:407] Connection header received
0809 04:44:43.846148 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:43.846153 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:43.846234 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:43.846333 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:43.847219 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:43.848260 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:43.850744 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:43.851493 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:43.851678 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:43.856337 (+ 4659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.1:42379: BlockingRecv error: recv got EOF from 127.12.234.1:42379 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 04:44:44 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:44 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:44.104985 23034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:44.095956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.3:55565)
0809 04:44:44.096502 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:44.096516 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:44.096625 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:44.097120 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:44.097126 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:44.097761 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:44.098071 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:44.098089 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:44.099133 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:44.099137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:44.100134 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:44.100147 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:44.100797 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:44.100802 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:44.100925 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:44.101446 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:44.101462 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:44.104749 (+ 3287us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":334,"threads_started":1}
W20260809 04:44:44.105099 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:44.096052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:55565 (local address 127.12.234.62:46413)
0809 04:44:44.096224 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:44.096228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:44.096588 (+ 360us) server_negotiation.cc:407] Connection header received
0809 04:44:44.096800 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:44.096805 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:44.096897 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:44.097002 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:44.098231 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:44.098990 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:44.100255 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:44.100644 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:44.100730 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:44.104903 (+ 4173us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.3:55565: BlockingRecv error: recv got EOF from 127.12.234.3:55565 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 09 04:44:44 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:44 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:44.445207 23035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:44.437282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.2:58233)
0809 04:44:44.437733 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:44.437747 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:44.437894 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:44.438199 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:44.438203 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:44.438595 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:44.438765 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:44.438774 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:44.439882 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:44.439885 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:44.440719 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:44.440729 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:44.441354 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:44.441359 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:44.441514 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:44.442055 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:44.442071 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:44.444906 (+ 2835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":238,"threads_started":1}
W20260809 04:44:44.445495 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:44.437337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:58233 (local address 127.12.234.62:46413)
0809 04:44:44.437549 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:44.437556 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:44.437824 (+ 268us) server_negotiation.cc:407] Connection header received
0809 04:44:44.438005 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:44.438010 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:44.438076 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:44.438165 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:44.438913 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:44.439705 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:44.440841 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:44.441208 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:44.441370 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:44.445076 (+ 3706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.2:58233: BlockingRecv error: recv got EOF from 127.12.234.2:58233 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 09 04:44:44 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:44 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 04:44:44.868160 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:44.857593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:54883 (local address 127.12.234.62:46413)
0809 04:44:44.857800 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:44.857808 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:44.858128 (+ 320us) server_negotiation.cc:407] Connection header received
0809 04:44:44.858257 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:44.858264 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:44.858361 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:44.858487 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:44.859234 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:44.860578 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:44.861770 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:44.862592 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:44.862732 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:44.867886 (+ 5154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.1:54883: BlockingRecv error: recv got EOF from 127.12.234.1:54883 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 04:44:44.868162 23036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:44.857506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.1:54883)
0809 04:44:44.858026 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:44.858043 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:44.858145 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:44.858524 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:44.858528 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:44.858914 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:44.859081 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:44.859088 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:44.860711 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:44.860715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:44.861634 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:44.861645 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:44.862746 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:44.862750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:44.862887 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:44.863549 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:44.863564 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:44.867694 (+ 4130us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":286,"threads_started":1}
Aug 09 04:44:45 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:45 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:45.114625 23037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:45.106059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.3:44937)
0809 04:44:45.106581 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:45.106597 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:45.106721 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:45.107017 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:45.107020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:45.107380 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:45.107533 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:45.107542 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:45.108468 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:45.108471 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:45.109415 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:45.109427 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:45.110070 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:45.110073 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:45.110189 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:45.110790 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:45.110806 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:45.114343 (+ 3537us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":294,"threads_started":1}
W20260809 04:44:45.114737 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:45.106189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:44937 (local address 127.12.234.62:46413)
0809 04:44:45.106358 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:45.106362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:45.106664 (+ 302us) server_negotiation.cc:407] Connection header received
0809 04:44:45.106833 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:45.106837 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:45.106895 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:45.106970 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:45.107656 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:45.108365 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:45.109541 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:45.109949 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:45.110035 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:45.114515 (+ 4480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.3:44937: BlockingRecv error: recv got EOF from 127.12.234.3:44937 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 04:44:45 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:45 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:45.454759 23038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:45.446153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.2:55653)
0809 04:44:45.446593 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:45.446608 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:45.446705 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:45.447028 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:45.447031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:45.447386 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:45.447552 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:45.447559 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:45.448493 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:45.448497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:45.449502 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:45.449513 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:45.450133 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:45.450137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:45.450321 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:45.451194 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:45.451210 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:45.454542 (+ 3332us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":256,"threads_started":1}
W20260809 04:44:45.454942 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:45.446236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:55653 (local address 127.12.234.62:46413)
0809 04:44:45.446482 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:45.446488 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:45.446680 (+ 192us) server_negotiation.cc:407] Connection header received
0809 04:44:45.446811 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:45.446814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:45.446900 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:45.446976 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:45.447659 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:45.448387 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:45.449619 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:45.450012 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:45.450157 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:45.454716 (+ 4559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.2:55653: BlockingRecv error: recv got EOF from 127.12.234.2:55653 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 09 04:44:45 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:45 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:45.877683 23039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:45.869135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.1:37651)
0809 04:44:45.869668 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:45.869684 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:45.869797 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:45.870081 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:45.870084 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:45.870503 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:45.870660 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:45.870668 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:45.871706 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:45.871712 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:45.872854 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:45.872866 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:45.873564 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:45.873570 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:45.873757 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:45.874290 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:45.874305 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:45.877386 (+ 3081us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":293,"threads_started":1}
W20260809 04:44:45.877765 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:45.869190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:37651 (local address 127.12.234.62:46413)
0809 04:44:45.869360 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:45.869365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:45.869757 (+ 392us) server_negotiation.cc:407] Connection header received
0809 04:44:45.869893 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:45.869897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:45.869957 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:45.870046 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:45.870776 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:45.871546 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:45.873000 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:45.873430 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:45.873568 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:45.877563 (+ 3995us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.1:37651: BlockingRecv error: recv got EOF from 127.12.234.1:37651 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 04:44:46 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:46 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:46.123961 23040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:46.115613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.3:46065)
0809 04:44:46.116113 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:46.116127 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:46.116228 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:46.116523 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:46.116526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:46.116903 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:46.117065 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:46.117072 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:46.117938 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:46.117942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:46.118959 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:46.118971 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:46.119720 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:46.119727 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:46.119946 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:46.120489 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:46.120504 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:46.123719 (+ 3215us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":292,"threads_started":1}
W20260809 04:44:46.124130 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:46.115704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:46065 (local address 127.12.234.62:46413)
0809 04:44:46.115934 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:46.115940 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:46.116200 (+ 260us) server_negotiation.cc:407] Connection header received
0809 04:44:46.116335 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:46.116338 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:46.116397 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:46.116466 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:46.117178 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:46.117822 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:46.119099 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:46.119552 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:46.119681 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:46.123901 (+ 4220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.3:46065: BlockingRecv error: recv got EOF from 127.12.234.3:46065 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 09 04:44:46 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:46 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:46.464298 23041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:46.455702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.2:47879)
0809 04:44:46.456228 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:46.456246 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:46.456404 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:46.456745 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:46.456749 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:46.457195 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:46.457340 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:46.457348 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:46.458339 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:46.458346 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:46.459288 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:46.459299 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:46.460031 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:46.460036 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:46.460236 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:46.460757 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:46.460774 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:46.463961 (+ 3187us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":264,"threads_started":1}
W20260809 04:44:46.464411 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:46.455978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:47879 (local address 127.12.234.62:46413)
0809 04:44:46.456222 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:46.456227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:46.456334 (+ 107us) server_negotiation.cc:407] Connection header received
0809 04:44:46.456521 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:46.456524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:46.456597 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:46.456710 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:46.457477 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:46.458200 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:46.459434 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:46.459898 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:46.460017 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:46.464188 (+ 4171us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.2:47879: BlockingRecv error: recv got EOF from 127.12.234.2:47879 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 09 04:44:46 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:46 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:46.890884 23042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:46.878689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.1:59495)
0809 04:44:46.879394 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:46.879410 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:46.879572 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:46.879937 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:46.879942 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:46.880397 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:46.880553 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:46.880560 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:46.881440 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:46.881445 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:46.882242 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:46.882251 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:46.882895 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:46.882900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:46.883066 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:46.886641 (+ 3575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:46.886659 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:46.890517 (+ 3858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":293,"threads_started":1}
W20260809 04:44:46.890985 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:46.878838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:59495 (local address 127.12.234.62:46413)
0809 04:44:46.879076 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:46.879082 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:46.879501 (+ 419us) server_negotiation.cc:407] Connection header received
0809 04:44:46.879709 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:46.879715 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:46.879787 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:46.879903 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:46.880688 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:46.881315 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:46.882382 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:46.882750 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:46.882876 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:46.890719 (+ 7843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.1:59495: BlockingRecv error: recv got EOF from 127.12.234.1:59495 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 09 04:44:47 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:47 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:47.133817 23043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:47.124957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.3:40033)
0809 04:44:47.125484 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:47.125499 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:47.125644 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:47.126010 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:47.126014 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:47.126459 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:47.126605 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:47.126614 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:47.127639 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:47.127645 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:47.128666 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:47.128679 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:47.129458 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:47.129463 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:47.129649 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:47.130229 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:47.130257 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:47.133581 (+ 3324us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":271,"threads_started":1}
W20260809 04:44:47.134066 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:47.124980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:40033 (local address 127.12.234.62:46413)
0809 04:44:47.125166 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:47.125174 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:47.125581 (+ 407us) server_negotiation.cc:407] Connection header received
0809 04:44:47.125807 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:47.125814 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:47.125879 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:47.125974 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:47.126745 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:47.127449 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:47.128818 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:47.129308 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:47.129445 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:47.133831 (+ 4386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.3:40033: BlockingRecv error: recv got EOF from 127.12.234.3:40033 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 04:44:47 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:47 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:47.474793 23044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:47.465200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.2:37093)
0809 04:44:47.465881 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:47.465895 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:47.466038 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:47.466353 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:47.466357 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:47.466753 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:47.466914 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:47.466922 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:47.467848 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:47.467855 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:47.468742 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:47.468757 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:47.469496 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:47.469502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:47.469672 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:47.470295 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:47.470310 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:47.474414 (+ 4104us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":420,"threads_started":1}
W20260809 04:44:47.474924 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:47.465251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:37093 (local address 127.12.234.62:46413)
0809 04:44:47.465616 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:47.465624 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:47.465971 (+ 347us) server_negotiation.cc:407] Connection header received
0809 04:44:47.466145 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:47.466151 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:47.466210 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:47.466301 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:47.467078 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:47.467726 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:47.468914 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:47.469325 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:47.469460 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:47.474655 (+ 5195us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.2:37093: BlockingRecv error: recv got EOF from 127.12.234.2:37093 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 04:44:47 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:47 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:47.900390 23045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:47.891902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.1:50257)
0809 04:44:47.892453 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:47.892468 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:47.892576 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:47.892853 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:47.892857 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:47.893229 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:47.893399 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:47.893407 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:47.894310 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:47.894313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:47.895268 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:47.895279 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:47.895917 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:47.895921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:47.896022 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:47.896532 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:47.896546 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:47.900177 (+ 3631us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":278,"threads_started":1}
W20260809 04:44:47.900535 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:47.892023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:50257 (local address 127.12.234.62:46413)
0809 04:44:47.892259 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:47.892267 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:47.892547 (+ 280us) server_negotiation.cc:407] Connection header received
0809 04:44:47.892668 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:47.892671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:47.892729 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:47.892801 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:47.893512 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:47.894205 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:47.895386 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:47.895796 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:47.895874 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:47.900339 (+ 4465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.1:50257: BlockingRecv error: recv got EOF from 127.12.234.1:50257 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 09 04:44:48 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:48 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:48.143882 23046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.134874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.3:34079)
0809 04:44:48.135405 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.135423 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:48.135528 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:48.135860 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:48.135863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:48.136370 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:48.136531 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:48.136561 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.137608 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.137611 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:48.138474 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:48.138485 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.139214 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.139218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:48.139341 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.139939 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:48.139957 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:48.143551 (+ 3594us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":293,"threads_started":1}
W20260809 04:44:48.143971 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:48.134988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:34079 (local address 127.12.234.62:46413)
0809 04:44:48.135188 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.135195 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.135502 (+ 307us) server_negotiation.cc:407] Connection header received
0809 04:44:48.135639 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.135644 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.135721 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.135821 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.136727 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.137475 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.138599 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.139061 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.139161 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.143781 (+ 4620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.3:34079: BlockingRecv error: recv got EOF from 127.12.234.3:34079 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 04:44:48 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:48 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:48.484962 23047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.476648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.2:34429)
0809 04:44:48.477199 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.477214 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:48.477336 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:48.477634 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:48.477637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:48.477979 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:48.478141 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:48.478149 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.479069 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.479073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:48.479995 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:48.480005 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.480717 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.480721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:48.480841 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.481369 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:48.481385 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:48.484754 (+ 3369us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":261,"threads_started":1}
W20260809 04:44:48.485121 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:48.476732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:34429 (local address 127.12.234.62:46413)
0809 04:44:48.476926 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.476930 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.477281 (+ 351us) server_negotiation.cc:407] Connection header received
0809 04:44:48.477427 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.477430 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.477486 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.477589 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.478253 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.478953 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.480218 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.480584 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.480665 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.484920 (+ 4255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.2:34429: BlockingRecv error: recv got EOF from 127.12.234.2:34429 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 09 04:44:48 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:48 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:48.918283 23048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:48.901664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.1:44247)
0809 04:44:48.902744 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:48.902769 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:48.903050 (+ 281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:48.903746 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:48.903753 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:48.904991 (+ 1238us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:48.905400 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:48.905431 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.907645 (+ 2214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.907654 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:48.909380 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:48.909402 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.910985 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.911001 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:48.911315 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.912415 (+ 1100us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:48.912444 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:48.917835 (+ 5391us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":837,"thread_start_us":506,"threads_started":1}
W20260809 04:44:48.918705 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:48.901911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:44247 (local address 127.12.234.62:46413)
0809 04:44:48.902331 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:48.902353 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:48.902887 (+ 534us) server_negotiation.cc:407] Connection header received
0809 04:44:48.903312 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:48.903339 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:48.903505 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:48.903702 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:48.905741 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.907398 (+ 1657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.909678 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:48.910637 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:48.910943 (+ 306us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:48.918232 (+ 7289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.1:44247: BlockingRecv error: recv got EOF from 127.12.234.1:44247 (error 108)
Metrics: {"server-negotiator.queue_time_us":134}
Aug 09 04:44:49 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:49 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:49.162536 23049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:49.145286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.3:56313)
0809 04:44:49.146409 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:49.146425 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:49.146596 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:49.147144 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:49.147148 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:49.147866 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:49.148218 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:49.148238 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:49.149907 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:49.149919 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:49.151775 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:49.151797 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:49.152970 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:49.152976 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:49.153159 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:49.153985 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:49.154031 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:49.161764 (+ 7733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":948,"thread_start_us":580,"threads_started":1}
W20260809 04:44:49.162633 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:49.145513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:56313 (local address 127.12.234.62:46413)
0809 04:44:49.145751 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:49.145755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:49.146522 (+ 767us) server_negotiation.cc:407] Connection header received
0809 04:44:49.146835 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:49.146847 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:49.146932 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:49.147065 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:49.148374 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:49.149646 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:49.152043 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:49.152778 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:49.152941 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:49.162103 (+ 9162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.3:56313: BlockingRecv error: recv got EOF from 127.12.234.3:56313 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 09 04:44:49 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:49 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:49.500948 23050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:49.486071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.2:44293)
0809 04:44:49.486901 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:49.486936 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:49.487115 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:49.487631 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:49.487634 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:49.488510 (+ 876us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:49.488848 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:49.488867 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:49.490684 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:49.490694 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:49.492577 (+ 1883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:49.492605 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:49.493632 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:49.493638 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:49.493866 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:49.494722 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:49.494762 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:49.500552 (+ 5790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":412,"threads_started":1}
W20260809 04:44:49.501123 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:49.486122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:44293 (local address 127.12.234.62:46413)
0809 04:44:49.486374 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:49.486379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:49.487062 (+ 683us) server_negotiation.cc:407] Connection header received
0809 04:44:49.487260 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:49.487280 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:49.487462 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:49.487598 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:49.489070 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:49.490464 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:49.492754 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:49.493368 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:49.493505 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:49.500781 (+ 7276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.2:44293: BlockingRecv error: recv got EOF from 127.12.234.2:44293 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 09 04:44:49 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:49 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:49.932871 23051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:49.919626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.1:42873)
0809 04:44:49.920427 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:49.920506 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:49.920714 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:49.921266 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:49.921274 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:49.922015 (+ 741us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:49.922360 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:49.922385 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:49.923988 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:49.923994 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:49.925366 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:49.925386 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:49.926398 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:49.926404 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:49.926603 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:49.927459 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:49.927490 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:49.932550 (+ 5060us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":416,"threads_started":1}
W20260809 04:44:49.933190 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:49.919666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:42873 (local address 127.12.234.62:46413)
0809 04:44:49.919884 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:49.919889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:49.920610 (+ 721us) server_negotiation.cc:407] Connection header received
0809 04:44:49.920915 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:49.920923 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:49.921063 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:49.921182 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:49.922582 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:49.923805 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:49.925523 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:49.926200 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:49.926358 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:49.932789 (+ 6431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.1:42873: BlockingRecv error: recv got EOF from 127.12.234.1:42873 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 09 04:44:50 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:50 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:50.181921 23052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:50.164844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.3:56417)
0809 04:44:50.165833 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:50.165887 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:50.166134 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:50.166688 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:50.166702 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:50.167696 (+ 994us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:50.168143 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:50.168172 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:50.169949 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:50.169957 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:50.172698 (+ 2741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:50.172735 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:50.174549 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:50.174571 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:50.174921 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:50.175805 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:50.175841 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:50.181241 (+ 5400us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":694,"thread_start_us":478,"threads_started":1}
W20260809 04:44:50.181954 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:50.164928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:56417 (local address 127.12.234.62:46413)
0809 04:44:50.165357 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:50.165381 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:50.166004 (+ 623us) server_negotiation.cc:407] Connection header received
0809 04:44:50.166342 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:50.166345 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:50.166463 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:50.166610 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:50.168386 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:50.169704 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:50.172975 (+ 3271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:50.174206 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:50.174441 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:50.181558 (+ 7117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.3:56417: BlockingRecv error: recv got EOF from 127.12.234.3:56417 (error 108)
Metrics: {"server-negotiator.queue_time_us":127}
Aug 09 04:44:50 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:50 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:50.516827 23053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:50.502575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.2:53411)
0809 04:44:50.503446 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:50.503500 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:50.503736 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:50.504591 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:50.504603 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:50.505908 (+ 1305us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:50.506162 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:50.506179 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:50.507530 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:50.507533 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:50.509142 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:50.509169 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:50.510259 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:50.510265 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:50.510476 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:50.511229 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:50.511265 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:50.516306 (+ 5041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":705,"thread_start_us":502,"threads_started":1}
W20260809 04:44:50.516983 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:50.502621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:53411 (local address 127.12.234.62:46413)
0809 04:44:50.502941 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:50.502967 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:50.503630 (+ 663us) server_negotiation.cc:407] Connection header received
0809 04:44:50.504063 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:50.504067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:50.504301 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:50.504455 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:50.506356 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:50.507365 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:50.509337 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:50.510069 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:50.510228 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:50.516571 (+ 6343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.2:53411: BlockingRecv error: recv got EOF from 127.12.234.2:53411 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 09 04:44:50 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:50 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:50.950964 23054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:50.934555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.1:47021)
0809 04:44:50.935407 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:50.935448 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:50.935707 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:50.936308 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:50.936314 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:50.937143 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:50.937434 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:50.937460 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:50.939116 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:50.939126 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:50.941043 (+ 1917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:50.941066 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:50.942398 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:50.942415 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:50.942903 (+ 488us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:50.943989 (+ 1086us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:50.944032 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:50.950200 (+ 6168us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":421,"threads_started":1}
W20260809 04:44:50.951086 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:50.934582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:47021 (local address 127.12.234.62:46413)
0809 04:44:50.934854 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:50.934864 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:50.935610 (+ 746us) server_negotiation.cc:407] Connection header received
0809 04:44:50.935944 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:50.935951 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:50.936086 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:50.936264 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:50.937771 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:50.938852 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:50.941300 (+ 2448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:50.942196 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:50.942387 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:50.950594 (+ 8207us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.1:47021: BlockingRecv error: recv got EOF from 127.12.234.1:47021 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 09 04:44:51 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:51 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:51.197155 23055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.183679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.3:38551)
0809 04:44:51.184612 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.184636 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:51.184799 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:51.185264 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:51.185267 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:51.185988 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:51.186258 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:51.186273 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.188110 (+ 1837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.188116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:51.189935 (+ 1819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:51.189960 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.191010 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.191028 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:51.191340 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.192039 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:51.192068 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:51.196807 (+ 4739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":770,"thread_start_us":545,"threads_started":1}
W20260809 04:44:51.197356 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:51.183790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:38551 (local address 127.12.234.62:46413)
0809 04:44:51.184227 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:51.184234 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:51.184727 (+ 493us) server_negotiation.cc:407] Connection header received
0809 04:44:51.184958 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:51.184971 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:51.185072 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:51.185221 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:51.186500 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.187816 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.190179 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.190785 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.191004 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.197011 (+ 6007us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.3:38551: BlockingRecv error: recv got EOF from 127.12.234.3:38551 (error 108)
Metrics: {"server-negotiator.queue_time_us":181}
Aug 09 04:44:51 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:51 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:51.532402 23056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.518246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.2:43255)
0809 04:44:51.519052 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.519083 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:51.519288 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:51.519730 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:51.519734 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:51.520521 (+ 787us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:51.520814 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:51.520833 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.522509 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.522515 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:51.524450 (+ 1935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:51.524472 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.525418 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.525423 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:51.525611 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.526470 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:51.526500 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:51.531879 (+ 5379us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":495,"threads_started":1}
W20260809 04:44:51.532509 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:51.518336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:43255 (local address 127.12.234.62:46413)
0809 04:44:51.518598 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:51.518610 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:51.519212 (+ 602us) server_negotiation.cc:407] Connection header received
0809 04:44:51.519451 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:51.519455 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:51.519545 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:51.519660 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:51.520982 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.522340 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.524615 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.525210 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.525369 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.532189 (+ 6820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.2:43255: BlockingRecv error: recv got EOF from 127.12.234.2:43255 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 09 04:44:51 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:51 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:51.964627 23057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:51.952401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.1:54551)
0809 04:44:51.952917 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:51.952933 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:51.953055 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:51.953350 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:51.953352 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:51.953746 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:51.953912 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:51.953921 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.954819 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.954823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:51.955648 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:51.955658 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.956306 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.956310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:51.956431 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.961322 (+ 4891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:51.961339 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:51.964410 (+ 3071us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":296,"threads_started":1}
W20260809 04:44:51.964784 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:51.952604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:54551 (local address 127.12.234.62:46413)
0809 04:44:51.952763 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:51.952768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:51.953014 (+ 246us) server_negotiation.cc:407] Connection header received
0809 04:44:51.953142 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:51.953145 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:51.953202 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:51.953294 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:51.954028 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.954718 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.955763 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:51.956188 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:51.956272 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:51.964587 (+ 8315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.1:54551: BlockingRecv error: recv got EOF from 127.12.234.1:54551 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 04:44:52 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:52 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:52.207394 23058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:52.198374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.3:36627)
0809 04:44:52.198859 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:52.198875 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:52.199026 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:52.199416 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:52.199423 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:52.199819 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:52.199984 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:52.199994 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:52.200996 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:52.201002 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:52.202021 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:52.202032 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:52.202707 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:52.202726 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:52.202884 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:52.203473 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:52.203490 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:52.207178 (+ 3688us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":252,"threads_started":1}
W20260809 04:44:52.207528 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:52.198461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:36627 (local address 127.12.234.62:46413)
0809 04:44:52.198679 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:52.198687 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:52.198960 (+ 273us) server_negotiation.cc:407] Connection header received
0809 04:44:52.199165 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:52.199169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:52.199247 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:52.199352 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:52.200142 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:52.200864 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:52.202158 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:52.202540 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:52.202689 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:52.207338 (+ 4649us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.3:36627: BlockingRecv error: recv got EOF from 127.12.234.3:36627 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 09 04:44:52 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:52 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:52.543046 23059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:52.533531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.2:54593)
0809 04:44:52.534086 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:52.534107 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:52.534261 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:52.534578 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:52.534581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:52.535016 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:52.535224 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:52.535233 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:52.536364 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:52.536368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:52.537416 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:52.537430 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:52.538260 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:52.538264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:52.538403 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:52.538999 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:52.539019 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:52.542691 (+ 3672us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":274,"threads_started":1}
W20260809 04:44:52.543143 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:52.533609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:54593 (local address 127.12.234.62:46413)
0809 04:44:52.533847 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:52.533855 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:52.534207 (+ 352us) server_negotiation.cc:407] Connection header received
0809 04:44:52.534372 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:52.534376 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:52.534448 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:52.534523 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:52.535345 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:52.536228 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:52.537548 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:52.538119 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:52.538207 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:52.542883 (+ 4676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.2:54593: BlockingRecv error: recv got EOF from 127.12.234.2:54593 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 09 04:44:52 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:52 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:52.974900 23060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:52.965658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.1:53785)
0809 04:44:52.966284 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:52.966302 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:52.966586 (+ 284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:52.967278 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:52.967283 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:52.967856 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:52.968099 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:52.968112 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:52.969068 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:52.969072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:52.970017 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:52.970028 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:52.970748 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:52.970752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:52.970877 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:52.971535 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:52.971553 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:52.974698 (+ 3145us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":341,"threads_started":1}
W20260809 04:44:52.975077 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:52.965709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:53785 (local address 127.12.234.62:46413)
0809 04:44:52.965875 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:52.965881 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:52.966395 (+ 514us) server_negotiation.cc:407] Connection header received
0809 04:44:52.966911 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:52.966916 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:52.966993 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:52.967087 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:52.968233 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:52.968927 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:52.970146 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:52.970569 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:52.970675 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:52.974864 (+ 4189us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.1:53785: BlockingRecv error: recv got EOF from 127.12.234.1:53785 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 04:44:53 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:53 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:53.218026 23061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:53.208492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.3:49167)
0809 04:44:53.208964 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:53.208980 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:53.209181 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:53.209554 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:53.209561 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:53.210026 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:53.210213 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:53.210221 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:53.211188 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:53.211194 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:53.212347 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:53.212360 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:53.213424 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:53.213428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:53.213622 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:53.214196 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:53.214278 (+ 82us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:53.217797 (+ 3519us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":257,"threads_started":1}
W20260809 04:44:53.218242 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:53.208544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:49167 (local address 127.12.234.62:46413)
0809 04:44:53.208785 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:53.208793 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:53.209080 (+ 287us) server_negotiation.cc:407] Connection header received
0809 04:44:53.209352 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:53.209356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:53.209416 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:53.209516 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:53.210384 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:53.211058 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:53.212524 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:53.213261 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:53.213409 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:53.217991 (+ 4582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.3:49167: BlockingRecv error: recv got EOF from 127.12.234.3:49167 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 09 04:44:53 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:53 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:53.552798 23062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:53.544008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.2:52161)
0809 04:44:53.544502 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:53.544516 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:53.544632 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:53.544933 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:53.544936 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:53.545384 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:53.545556 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:53.545565 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:53.546462 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:53.546465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:53.547311 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:53.547321 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:53.547933 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:53.547936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:53.548098 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:53.548713 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:53.548730 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:53.552559 (+ 3829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":301,"threads_started":1}
W20260809 04:44:53.553006 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:53.544121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:52161 (local address 127.12.234.62:46413)
0809 04:44:53.544328 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:53.544336 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:53.544603 (+ 267us) server_negotiation.cc:407] Connection header received
0809 04:44:53.544728 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:53.544731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:53.544811 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:53.544880 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:53.545669 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:53.546332 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:53.547431 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:53.547820 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:53.547903 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:53.552743 (+ 4840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.2:52161: BlockingRecv error: recv got EOF from 127.12.234.2:52161 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 09 04:44:53 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:53 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:53.984184 23063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:53.975804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.1:58969)
0809 04:44:53.976288 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:53.976305 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:53.976410 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:53.976677 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:53.976680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:53.977075 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:53.977217 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:53.977225 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:53.978250 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:53.978253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:53.979096 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:53.979106 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:53.979776 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:53.979780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:53.979899 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:53.980542 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:53.980560 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:53.983874 (+ 3314us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":272,"threads_started":1}
W20260809 04:44:53.984382 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:53.975905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:58969 (local address 127.12.234.62:46413)
0809 04:44:53.976109 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:53.976115 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:53.976374 (+ 259us) server_negotiation.cc:407] Connection header received
0809 04:44:53.976504 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:53.976507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:53.976565 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:53.976642 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:53.977329 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:53.978127 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:53.979211 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:53.979658 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:53.979747 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:53.984093 (+ 4346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.1:58969: BlockingRecv error: recv got EOF from 127.12.234.1:58969 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 04:44:54 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:54 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:54.228118 23064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:54.218989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.3:40327)
0809 04:44:54.219568 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:54.219586 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:54.219757 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:54.220199 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:54.220206 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:54.220675 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:54.220848 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:54.220856 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:54.221952 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:54.221955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:54.222912 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:54.222923 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:54.223596 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:54.223600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:54.223724 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:54.224291 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:54.224308 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:54.227722 (+ 3414us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":273,"threads_started":1}
W20260809 04:44:54.228268 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:54.219049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:40327 (local address 127.12.234.62:46413)
0809 04:44:54.219284 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:54.219291 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:54.219697 (+ 406us) server_negotiation.cc:407] Connection header received
0809 04:44:54.219915 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:54.219920 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:54.220005 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:54.220100 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:54.220976 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:54.221836 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:54.223029 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:54.223418 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:54.223503 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:54.228010 (+ 4507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.3:40327: BlockingRecv error: recv got EOF from 127.12.234.3:40327 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 09 04:44:54 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:54 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:54.562345 23065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:54.553656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.2:51083)
0809 04:44:54.554149 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:54.554165 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:54.554277 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:54.554577 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:54.554580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:54.554962 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:54.555106 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:54.555114 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:54.556130 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:54.556133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:54.556979 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:54.556988 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:54.557769 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:54.557774 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:54.557949 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:54.558560 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:54.558577 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:54.562007 (+ 3430us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":283,"threads_started":1}
W20260809 04:44:54.562425 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:54.553877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:51083 (local address 127.12.234.62:46413)
0809 04:44:54.554110 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:54.554118 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:54.554250 (+ 132us) server_negotiation.cc:407] Connection header received
0809 04:44:54.554377 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:54.554381 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:54.554459 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:54.554546 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:54.555235 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:54.555991 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:54.557132 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:54.557623 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:54.557752 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:54.562186 (+ 4434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.2:51083: BlockingRecv error: recv got EOF from 127.12.234.2:51083 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 09 04:44:54 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:54 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.1@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:54.995615 23066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:54.985537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.1:59239)
0809 04:44:54.986057 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:54.986072 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:54.986235 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:54.986583 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:54.986590 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:54.987030 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:54.987306 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:54.987316 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:54.988400 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:54.988406 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:54.989698 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:54.989715 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:54.990749 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:54.990756 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:54.990993 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:54.991710 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:54.991729 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:54.995262 (+ 3533us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":300,"threads_started":1}
W20260809 04:44:54.995682 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:54.985618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:59239 (local address 127.12.234.62:46413)
0809 04:44:54.985889 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:54.985897 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:54.986160 (+ 263us) server_negotiation.cc:407] Connection header received
0809 04:44:54.986371 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:54.986376 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:54.986434 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:54.986531 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:54.987447 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:54.988207 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:54.989863 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:54.990511 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:54.990645 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:54.995440 (+ 4795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.1:59239: BlockingRecv error: recv got EOF from 127.12.234.1:59239 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 09 04:44:55 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:55 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.3@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:55.238376 23067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:55.229189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.3:48803)
0809 04:44:55.229722 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:55.229738 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:55.229890 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:55.230253 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:55.230260 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:55.230718 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:55.230861 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:55.230869 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:55.231931 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:55.231936 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:55.232856 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:55.232868 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:55.233567 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:55.233573 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:55.233757 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:55.234379 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:55.234397 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:55.238158 (+ 3761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":294,"threads_started":1}
W20260809 04:44:55.238737 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:55.229273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:48803 (local address 127.12.234.62:46413)
0809 04:44:55.229617 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:55.229623 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:55.229824 (+ 201us) server_negotiation.cc:407] Connection header received
0809 04:44:55.230032 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:55.230037 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:55.230112 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:55.230214 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:55.231019 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:55.231778 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:55.232999 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:55.233421 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:55.233562 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:55.238386 (+ 4824us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.3:48803: BlockingRecv error: recv got EOF from 127.12.234.3:48803 (error 108)
Metrics: {"server-negotiator.queue_time_us":198}
Aug 09 04:44:55 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 04:44:55 dist-test-slave-1jjb krb5kdc[21934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.234.2@KRBTEST.COM for oryx/127.12.234.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 04:44:55.573246 23068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:55.563840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46413 (local address 127.12.234.2:45749)
0809 04:44:55.564681 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:55.564697 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:55.564839 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:55.565359 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:55.565363 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:55.565771 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:55.565978 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:55.565988 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:55.566906 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:55.566921 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:55.567746 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:55.567757 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:55.568528 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:55.568533 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:55.568706 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:55.569363 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:44:55.569382 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:55.572978 (+ 3596us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.234.62:46413: Server oryx/127.12.234.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":375,"threads_started":1}
W20260809 04:44:55.573402 22734 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:55.563988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:45749 (local address 127.12.234.62:46413)
0809 04:44:55.564200 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:55.564207 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:55.564781 (+ 574us) server_negotiation.cc:407] Connection header received
0809 04:44:55.564980 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:55.564984 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:55.565101 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:55.565200 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:55.566101 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:55.566790 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:55.567892 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:55.568380 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:55.568476 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:55.573169 (+ 4693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.234.2:45749: BlockingRecv error: recv got EOF from 127.12.234.2:45749 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 04:44:55.975045 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 22571
I20260809 04:44:55.989774 23069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:55.976306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39500 (local address 127.12.234.1:42133)
0809 04:44:55.976949 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:55.976960 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:55.977680 (+ 720us) server_negotiation.cc:407] Connection header received
0809 04:44:55.977863 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:55.977870 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:55.978106 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:55.978268 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:55.980282 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:55.981246 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:55.982370 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:55.982844 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:55.982939 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:55.986327 (+ 3388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:55.986335 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:55.986347 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:55.986398 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:55.989286 (+ 2888us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.12.234.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 04:44:55.989568 (+ 282us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39500: Request ticket server kudu/127.12.234.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":282,"threads_started":1}
W20260809 04:44:55.989868 23069 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39500: Request ticket server kudu/127.12.234.1@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 04:44:55.990386 13224 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 22571: Not authorized: Client connection negotiation failed: client connection to 127.12.234.1:42133: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.12.234.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 04:44:55.990645 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 22571
I20260809 04:44:56.013749 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 22735
I20260809 04:44:56.028064 23071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:56.014923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58050 (local address 127.12.234.2:42603)
0809 04:44:56.015740 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:56.015747 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:56.015776 (+ 29us) server_negotiation.cc:407] Connection header received
0809 04:44:56.015838 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:56.015845 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:56.016202 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:56.016460 (+ 258us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:56.017985 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:56.019316 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:56.020860 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:56.021432 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:56.021614 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:56.024509 (+ 2895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:56.024519 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:56.024532 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:56.024580 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:56.027355 (+ 2775us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.12.234.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 04:44:56.027746 (+ 391us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58050: Request ticket server kudu/127.12.234.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":492,"threads_started":1}
W20260809 04:44:56.028050 13224 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 22735: Not authorized: Client connection negotiation failed: client connection to 127.12.234.2:42603: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.12.234.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 04:44:56.028234 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 22735
W20260809 04:44:56.028219 23071 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58050: Request ticket server kudu/127.12.234.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 04:44:56.050076 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 22878
W20260809 04:44:56.064327 13224 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 22878: Not authorized: Client connection negotiation failed: client connection to 127.12.234.3:39357: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.12.234.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 04:44:56.064467 23072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:56.051259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33686 (local address 127.12.234.3:39357)
0809 04:44:56.052112 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:56.052123 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:56.052175 (+ 52us) server_negotiation.cc:407] Connection header received
0809 04:44:56.052254 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:56.052264 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:56.052624 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:56.052851 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:56.054329 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:56.056393 (+ 2064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:56.057466 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:56.057999 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:56.058103 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:56.060898 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:56.060907 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:56.060919 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:44:56.060968 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:56.063619 (+ 2651us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.12.234.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 04:44:56.063955 (+ 336us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33686: Request ticket server kudu/127.12.234.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":433,"threads_started":1}
I20260809 04:44:56.064543 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 22878
I20260809 04:44:56.085003 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 22493
I20260809 04:44:56.086670 22564 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:56.212206 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 22493
2026-08-09T04:44:56Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21477 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T04:44:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:56Z Disabled control of system clock
I20260809 04:44:56.305104 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:34401
--webserver_interface=127.12.234.62
--webserver_port=0
--builtin_ntp_servers=127.12.234.20:42607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:34401
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 04:44:56.629544 23079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:56.629870 23079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:56.630036 23079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:56.637228 23079 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:56.637343 23079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:56.637413 23079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:56.637508 23079 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:56.637586 23079 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:56.645970 23079 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:42607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:34401
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:34401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:56.647327 23079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:56.649214 23079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:56.660549 23084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:56.660766 23087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:56.660848 23085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:56.661619 23079 server_base.cc:1034] running on GCE node
I20260809 04:44:56.662257 23079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:56.663395 23079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:56.664587 23079 hybrid_clock.cc:648] HybridClock initialized: now 1786250696664543 us; error 33 us; skew 500 ppm
I20260809 04:44:56.664899 23079 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:44:56.667083 23079 webserver.cc:460] Webserver started at http://127.12.234.62:35609/ using document root <none> and password file <none>
I20260809 04:44:56.667773 23079 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:56.667872 23079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:56.668262 23079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:56.671303 23079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "d02b7ee6edba41f49a6eb63364ce8375"
format_stamp: "Formatted at 2026-08-09 04:44:56 on dist-test-slave-1jjb"
server_key: "b3bb98979f87a0a47719bea30505c987"
server_key_iv: "e4fb192abbdb54fc5aa6aefc8ca64544"
server_key_version: "encryptionkey@0"
I20260809 04:44:56.672061 23079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "d02b7ee6edba41f49a6eb63364ce8375"
format_stamp: "Formatted at 2026-08-09 04:44:56 on dist-test-slave-1jjb"
server_key: "b3bb98979f87a0a47719bea30505c987"
server_key_iv: "e4fb192abbdb54fc5aa6aefc8ca64544"
server_key_version: "encryptionkey@0"
I20260809 04:44:56.677083 23079 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 04:44:56.680248 23093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:56.681493 23079 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:44:56.681684 23079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "d02b7ee6edba41f49a6eb63364ce8375"
format_stamp: "Formatted at 2026-08-09 04:44:56 on dist-test-slave-1jjb"
server_key: "b3bb98979f87a0a47719bea30505c987"
server_key_iv: "e4fb192abbdb54fc5aa6aefc8ca64544"
server_key_version: "encryptionkey@0"
I20260809 04:44:56.681902 23079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:56.706144 23079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:56.706876 23079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:56.707142 23079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:56.727224 23079 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:34401
I20260809 04:44:56.727273 23150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:34401 every 8 connection(s)
I20260809 04:44:56.728564 23079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:56.732430 23151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:56.736896 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 23079
I20260809 04:44:56.737104 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:44:56.738152 13224 external_mini_cluster.cc:1428] Setting key 9991b2bdb5ad8a8e5d3394892f2fe3ad
I20260809 04:44:56.743682 23151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d02b7ee6edba41f49a6eb63364ce8375: Bootstrap starting.
I20260809 04:44:56.746562 23151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d02b7ee6edba41f49a6eb63364ce8375: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:56.747467 23151 log.cc:824] T 00000000000000000000000000000000 P d02b7ee6edba41f49a6eb63364ce8375: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:56.750217 23151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d02b7ee6edba41f49a6eb63364ce8375: No bootstrap required, opened a new log
I20260809 04:44:56.751628 23154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:56.740387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35478 (local address 127.12.234.62:34401)
0809 04:44:56.741393 (+ 1006us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:56.741402 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:56.741443 (+ 41us) server_negotiation.cc:407] Connection header received
0809 04:44:56.743080 (+ 1637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:56.743100 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:56.743573 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:56.743930 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:56.744761 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:56.745941 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:56.747356 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:56.748149 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:56.748363 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:56.749553 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:56.749561 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:56.749593 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:56.750101 (+ 508us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:56.750172 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:56.750193 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:56.750330 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:56.750566 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 04:44:56.750854 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":422,"threads_started":1}
I20260809 04:44:56.757314 23151 raft_consensus.cc:348] T 00000000000000000000000000000000 P d02b7ee6edba41f49a6eb63364ce8375 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02b7ee6edba41f49a6eb63364ce8375" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 34401 } }
I20260809 04:44:56.757656 23151 raft_consensus.cc:374] T 00000000000000000000000000000000 P d02b7ee6edba41f49a6eb63364ce8375 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:56.757757 23151 raft_consensus.cc:729] T 00000000000000000000000000000000 P d02b7ee6edba41f49a6eb63364ce8375 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d02b7ee6edba41f49a6eb63364ce8375, State: Initialized, Role: FOLLOWER
I20260809 04:44:56.758205 23151 consensus_queue.cc:260] T 00000000000000000000000000000000 P d02b7ee6edba41f49a6eb63364ce8375 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02b7ee6edba41f49a6eb63364ce8375" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 34401 } }
I20260809 04:44:56.758409 23151 raft_consensus.cc:388] T 00000000000000000000000000000000 P d02b7ee6edba41f49a6eb63364ce8375 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:56.758533 23151 raft_consensus.cc:482] T 00000000000000000000000000000000 P d02b7ee6edba41f49a6eb63364ce8375 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:56.758687 23151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d02b7ee6edba41f49a6eb63364ce8375 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:56.760978 23151 raft_consensus.cc:504] T 00000000000000000000000000000000 P d02b7ee6edba41f49a6eb63364ce8375 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02b7ee6edba41f49a6eb63364ce8375" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 34401 } }
I20260809 04:44:56.761443 23151 leader_election.cc:302] T 00000000000000000000000000000000 P d02b7ee6edba41f49a6eb63364ce8375 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d02b7ee6edba41f49a6eb63364ce8375; no voters:
I20260809 04:44:56.762051 23151 leader_election.cc:288] T 00000000000000000000000000000000 P d02b7ee6edba41f49a6eb63364ce8375 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:56.762179 23156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d02b7ee6edba41f49a6eb63364ce8375 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:56.762969 23156 raft_consensus.cc:686] T 00000000000000000000000000000000 P d02b7ee6edba41f49a6eb63364ce8375 [term 1 LEADER]: Becoming Leader. State: Replica: d02b7ee6edba41f49a6eb63364ce8375, State: Running, Role: LEADER
I20260809 04:44:56.763464 23156 consensus_queue.cc:237] T 00000000000000000000000000000000 P d02b7ee6edba41f49a6eb63364ce8375 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02b7ee6edba41f49a6eb63364ce8375" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 34401 } }
I20260809 04:44:56.764037 23151 sys_catalog.cc:564] T 00000000000000000000000000000000 P d02b7ee6edba41f49a6eb63364ce8375 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:56.766069 23158 sys_catalog.cc:455] T 00000000000000000000000000000000 P d02b7ee6edba41f49a6eb63364ce8375 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d02b7ee6edba41f49a6eb63364ce8375. Latest consensus state: current_term: 1 leader_uuid: "d02b7ee6edba41f49a6eb63364ce8375" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02b7ee6edba41f49a6eb63364ce8375" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 34401 } } }
I20260809 04:44:56.766434 23158 sys_catalog.cc:458] T 00000000000000000000000000000000 P d02b7ee6edba41f49a6eb63364ce8375 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:56.766247 23157 sys_catalog.cc:455] T 00000000000000000000000000000000 P d02b7ee6edba41f49a6eb63364ce8375 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d02b7ee6edba41f49a6eb63364ce8375" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d02b7ee6edba41f49a6eb63364ce8375" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 34401 } } }
I20260809 04:44:56.766649 23157 sys_catalog.cc:458] T 00000000000000000000000000000000 P d02b7ee6edba41f49a6eb63364ce8375 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:56.768678 23162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:56.773759 23162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:56.781111 23162 catalog_manager.cc:1347] Generated new cluster ID: 8689f0326a0e4afea54c7aa466fed806
I20260809 04:44:56.781198 23162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:56.799019 23162 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:56.800251 23162 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:56.805919 23162 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d02b7ee6edba41f49a6eb63364ce8375: Generated new TSK 0
I20260809 04:44:56.806583 23162 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:44:56.823169 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:0
--local_ip_for_outbound_sockets=127.12.234.1
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:34401
--builtin_ntp_servers=127.12.234.20:42607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 04:44:57.140331 23177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:57.140609 23177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:57.140781 23177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:57.147384 23177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:57.147486 23177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:57.147645 23177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:44:57.156317 23177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:42607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:34401
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:57.157361 23177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:57.158998 23177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:57.171161 23182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:57.171258 23185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:57.171303 23183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:57.172344 23177 server_base.cc:1034] running on GCE node
I20260809 04:44:57.172976 23177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:57.173786 23177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:57.175001 23177 hybrid_clock.cc:648] HybridClock initialized: now 1786250697174959 us; error 54 us; skew 500 ppm
I20260809 04:44:57.175275 23177 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:44:57.182145 23177 webserver.cc:460] Webserver started at http://127.12.234.1:40331/ using document root <none> and password file <none>
I20260809 04:44:57.182772 23177 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:57.182906 23177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:57.183243 23177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:57.185971 23177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/data/instance:
uuid: "e484913a063442839eef7141a058e51d"
format_stamp: "Formatted at 2026-08-09 04:44:57 on dist-test-slave-1jjb"
server_key: "3c385d4de894c1a66aac44f22fd5d6bd"
server_key_iv: "e41420f959bd79b9e571f01ebec6cf9e"
server_key_version: "encryptionkey@0"
I20260809 04:44:57.186748 23177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance:
uuid: "e484913a063442839eef7141a058e51d"
format_stamp: "Formatted at 2026-08-09 04:44:57 on dist-test-slave-1jjb"
server_key: "3c385d4de894c1a66aac44f22fd5d6bd"
server_key_iv: "e41420f959bd79b9e571f01ebec6cf9e"
server_key_version: "encryptionkey@0"
I20260809 04:44:57.191998 23177 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 04:44:57.195123 23191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:57.196381 23177 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260809 04:44:57.196547 23177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "e484913a063442839eef7141a058e51d"
format_stamp: "Formatted at 2026-08-09 04:44:57 on dist-test-slave-1jjb"
server_key: "3c385d4de894c1a66aac44f22fd5d6bd"
server_key_iv: "e41420f959bd79b9e571f01ebec6cf9e"
server_key_version: "encryptionkey@0"
I20260809 04:44:57.196741 23177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:57.208554 23177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:57.209322 23177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:57.209610 23177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:57.210532 23177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:57.211913 23177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:57.212030 23177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:57.212209 23177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:57.212309 23177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:57.246490 23177 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:35013
I20260809 04:44:57.246548 23311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:35013 every 8 connection(s)
I20260809 04:44:57.247768 23177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:44:57.249657 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 23177
I20260809 04:44:57.249856 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance
I20260809 04:44:57.250905 13224 external_mini_cluster.cc:1428] Setting key 16127767c2beeb8c40866ed805fffc97
I20260809 04:44:57.253611 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:0
--local_ip_for_outbound_sockets=127.12.234.2
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:34401
--builtin_ntp_servers=127.12.234.20:42607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 04:44:57.259958 23154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:57.250476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:42961 (local address 127.12.234.62:34401)
0809 04:44:57.250733 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:57.250740 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:57.252359 (+ 1619us) server_negotiation.cc:407] Connection header received
0809 04:44:57.254004 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:57.254009 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:57.254082 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:57.254189 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:57.255089 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:57.255761 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:57.257297 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:57.257694 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:57.257818 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:57.259147 (+ 1329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:57.259152 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:57.259157 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:57.259181 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:57.259209 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:57.259220 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:57.259324 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:57.259534 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 04:44:57.259641 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 04:44:57.260897 23314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:57.251069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:34401 (local address 127.12.234.1:42961)
0809 04:44:57.252216 (+ 1147us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:57.252256 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:57.253761 (+ 1505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:57.254395 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:57.254402 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:57.254472 (+ 70us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:57.254948 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:57.254962 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:57.255920 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:57.255923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:57.257156 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:57.257167 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:57.257912 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:57.257921 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:57.258122 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:57.258855 (+ 733us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:44:57.258857 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:57.258890 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:44:57.258955 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:57.259323 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:57.259334 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:57.259343 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 04:44:57.259518 (+ 175us) client_negotiation.cc:240] Negotiation successful
0809 04:44:57.259887 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":360,"threads_started":1}
I20260809 04:44:57.262135 23312 heartbeater.cc:347] Connected to a master server at 127.12.234.62:34401
I20260809 04:44:57.262405 23312 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:57.263182 23312 heartbeater.cc:511] Master 127.12.234.62:34401 requested a full tablet report, sending...
I20260809 04:44:57.265465 23111 ts_manager.cc:194] Registered new tserver with Master: e484913a063442839eef7141a058e51d (127.12.234.1:35013)
I20260809 04:44:57.266877 23111 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.234.1:42961
W20260809 04:44:57.603376 23315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:57.603612 23315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:57.603736 23315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:57.610932 23315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:57.611018 23315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:57.611167 23315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:44:57.620690 23315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:42607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:34401
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:57.621902 23315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:57.623895 23315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:57.637778 23320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:57.637835 23321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:57.637791 23323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:57.638999 23315 server_base.cc:1034] running on GCE node
I20260809 04:44:57.639631 23315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:57.640766 23315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:57.642040 23315 hybrid_clock.cc:648] HybridClock initialized: now 1786250697641989 us; error 39 us; skew 500 ppm
I20260809 04:44:57.642426 23315 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:44:57.644745 23315 webserver.cc:460] Webserver started at http://127.12.234.2:45013/ using document root <none> and password file <none>
I20260809 04:44:57.645485 23315 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:57.645623 23315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:57.645991 23315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:57.648696 23315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/data/instance:
uuid: "c27de1ccd34c48aa93981dbf8a7622f8"
format_stamp: "Formatted at 2026-08-09 04:44:57 on dist-test-slave-1jjb"
server_key: "3ee8a78398719ea79014d6fe8dea126e"
server_key_iv: "bfb13d1118af2c93bad24cf469f34f83"
server_key_version: "encryptionkey@0"
I20260809 04:44:57.649560 23315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance:
uuid: "c27de1ccd34c48aa93981dbf8a7622f8"
format_stamp: "Formatted at 2026-08-09 04:44:57 on dist-test-slave-1jjb"
server_key: "3ee8a78398719ea79014d6fe8dea126e"
server_key_iv: "bfb13d1118af2c93bad24cf469f34f83"
server_key_version: "encryptionkey@0"
I20260809 04:44:57.654635 23315 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 04:44:57.658243 23329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:57.659509 23315 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:44:57.659731 23315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "c27de1ccd34c48aa93981dbf8a7622f8"
format_stamp: "Formatted at 2026-08-09 04:44:57 on dist-test-slave-1jjb"
server_key: "3ee8a78398719ea79014d6fe8dea126e"
server_key_iv: "bfb13d1118af2c93bad24cf469f34f83"
server_key_version: "encryptionkey@0"
I20260809 04:44:57.659945 23315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:57.690351 23315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:57.691130 23315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:57.691403 23315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:57.692286 23315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:57.693677 23315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:57.693794 23315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:57.693921 23315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:57.694012 23315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:57.734565 23315 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:33563
I20260809 04:44:57.734627 23450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:33563 every 8 connection(s)
I20260809 04:44:57.735783 23315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:44:57.743422 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 23315
I20260809 04:44:57.743623 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance
I20260809 04:44:57.744208 13224 external_mini_cluster.cc:1428] Setting key 14c28da9b25bb48dba3efcd4a7c03844
I20260809 04:44:57.747092 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:0
--local_ip_for_outbound_sockets=127.12.234.3
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:34401
--builtin_ntp_servers=127.12.234.20:42607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 04:44:57.749521 23154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:57.738339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:34699 (local address 127.12.234.62:34401)
0809 04:44:57.738544 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:57.738550 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:57.740535 (+ 1985us) server_negotiation.cc:407] Connection header received
0809 04:44:57.742388 (+ 1853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:57.742395 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:57.742471 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:57.742591 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:57.743520 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:57.744597 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:57.746843 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:57.747301 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:57.747438 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:57.748708 (+ 1270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:57.748713 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:57.748718 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:57.748735 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:57.748772 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:57.748782 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:57.748890 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:57.749129 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 04:44:57.749223 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 04:44:57.750571 23453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:57.738870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:34401 (local address 127.12.234.2:34699)
0809 04:44:57.740367 (+ 1497us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:57.740418 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:57.742113 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:57.742765 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:57.742772 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:57.742842 (+ 70us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:57.743366 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:57.743380 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:57.745025 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:57.745031 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:57.746704 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:57.746720 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:57.747520 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:57.747527 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:57.747751 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:57.748422 (+ 671us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:44:57.748424 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:57.748459 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:44:57.748524 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:57.748892 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:57.748905 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:57.748914 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 04:44:57.749118 (+ 204us) client_negotiation.cc:240] Negotiation successful
0809 04:44:57.749477 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":928,"thread_start_us":753,"threads_started":1}
I20260809 04:44:57.751940 23451 heartbeater.cc:347] Connected to a master server at 127.12.234.62:34401
I20260809 04:44:57.752250 23451 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:57.752955 23451 heartbeater.cc:511] Master 127.12.234.62:34401 requested a full tablet report, sending...
I20260809 04:44:57.754273 23111 ts_manager.cc:194] Registered new tserver with Master: c27de1ccd34c48aa93981dbf8a7622f8 (127.12.234.2:33563)
I20260809 04:44:57.755499 23111 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.234.2:34699
W20260809 04:44:58.062408 23454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:58.062649 23454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:58.062762 23454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:58.069406 23454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:58.069489 23454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:58.069593 23454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:44:58.077582 23454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:42607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:34401
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:58.078626 23454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:58.080273 23454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:58.093258 23459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:58.093302 23460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:58.093214 23463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:58.097385 23454 server_base.cc:1034] running on GCE node
I20260809 04:44:58.097888 23454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:58.098699 23454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:58.099915 23454 hybrid_clock.cc:648] HybridClock initialized: now 1786250698099860 us; error 35 us; skew 500 ppm
I20260809 04:44:58.100229 23454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:44:58.102254 23454 webserver.cc:460] Webserver started at http://127.12.234.3:39993/ using document root <none> and password file <none>
I20260809 04:44:58.102829 23454 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:58.102911 23454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:58.103174 23454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:58.105847 23454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/data/instance:
uuid: "b7832f3286c648c4bb0c391d64fe8508"
format_stamp: "Formatted at 2026-08-09 04:44:58 on dist-test-slave-1jjb"
server_key: "3ed029c8a1bc987ee4ca12b823799933"
server_key_iv: "3a01d880018aafdeefae5d4901ade97c"
server_key_version: "encryptionkey@0"
I20260809 04:44:58.106504 23454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance:
uuid: "b7832f3286c648c4bb0c391d64fe8508"
format_stamp: "Formatted at 2026-08-09 04:44:58 on dist-test-slave-1jjb"
server_key: "3ed029c8a1bc987ee4ca12b823799933"
server_key_iv: "3a01d880018aafdeefae5d4901ade97c"
server_key_version: "encryptionkey@0"
I20260809 04:44:58.111052 23454 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 04:44:58.114027 23469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:58.115128 23454 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 04:44:58.115309 23454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "b7832f3286c648c4bb0c391d64fe8508"
format_stamp: "Formatted at 2026-08-09 04:44:58 on dist-test-slave-1jjb"
server_key: "3ed029c8a1bc987ee4ca12b823799933"
server_key_iv: "3a01d880018aafdeefae5d4901ade97c"
server_key_version: "encryptionkey@0"
I20260809 04:44:58.115501 23454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:58.139842 23454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:58.140666 23454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:58.140969 23454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:58.141808 23454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:58.143026 23454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:58.143139 23454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:58.143260 23454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:58.143350 23454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:58.176354 23454 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:35945
I20260809 04:44:58.176460 23592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:35945 every 8 connection(s)
I20260809 04:44:58.177744 23454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:44:58.185269 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 23454
I20260809 04:44:58.185422 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance
I20260809 04:44:58.185897 13224 external_mini_cluster.cc:1428] Setting key 14fa03e28b96b254cee038920953b319
I20260809 04:44:58.190702 23154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:58.180320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:43663 (local address 127.12.234.62:34401)
0809 04:44:58.180492 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:58.180497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:58.181866 (+ 1369us) server_negotiation.cc:407] Connection header received
0809 04:44:58.183714 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:58.183718 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:58.183793 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:58.183895 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:58.184908 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:58.185834 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:58.187858 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:58.188534 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:58.188674 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:58.190029 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:58.190034 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:58.190037 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:58.190055 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:58.190089 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:58.190102 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:58.190181 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:58.190385 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 04:44:58.190463 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 04:44:58.191563 23595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:58.180822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:34401 (local address 127.12.234.3:43663)
0809 04:44:58.181673 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:58.181706 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:58.183451 (+ 1745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:58.184137 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:58.184145 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:58.184232 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:58.184757 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:58.184773 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:58.186006 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:58.186012 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:58.187687 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:44:58.187702 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:44:58.188739 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:44:58.188744 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:44:58.188940 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:58.189750 (+ 810us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:44:58.189752 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:58.189788 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:44:58.189861 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:58.190193 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:58.190200 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:58.190206 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 04:44:58.190343 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 04:44:58.190595 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":277,"threads_started":1}
I20260809 04:44:58.192901 23593 heartbeater.cc:347] Connected to a master server at 127.12.234.62:34401
I20260809 04:44:58.193194 23593 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:58.193850 23593 heartbeater.cc:511] Master 127.12.234.62:34401 requested a full tablet report, sending...
I20260809 04:44:58.195070 23111 ts_manager.cc:194] Registered new tserver with Master: b7832f3286c648c4bb0c391d64fe8508 (127.12.234.3:35945)
I20260809 04:44:58.195842 23111 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.234.3:43663
I20260809 04:44:58.201736 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 04:44:58.208200 23154 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:44:58.206462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35494 (local address 127.12.234.62:34401)
0809 04:44:58.206615 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:58.206619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:58.207169 (+ 550us) server_negotiation.cc:407] Connection header received
0809 04:44:58.207286 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:58.207290 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:58.207421 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:58.207515 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:58.207935 (+ 420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35494: BlockingRecv error: recv got EOF from 127.0.0.1:35494 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:44:58.208887 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 23177
I20260809 04:44:58.216912 23603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:58.209973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44320 (local address 127.12.234.1:35013)
0809 04:44:58.210498 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:58.210505 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:58.210540 (+ 35us) server_negotiation.cc:407] Connection header received
0809 04:44:58.210591 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:58.210597 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:58.210781 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:58.210941 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:58.211335 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:58.212541 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:58.213763 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:58.214285 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:58.214423 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:58.215422 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:58.215430 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:58.215442 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:58.215996 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:58.216051 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:58.216134 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:58.216280 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:58.216435 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 04:44:58.216561 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":235,"threads_started":1}
I20260809 04:44:58.218154 23307 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:58.297797 23312 heartbeater.cc:503] Master 127.12.234.62:34401 was elected leader, sending a full tablet report...
I20260809 04:44:58.299250 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 23177
W20260809 04:44:58.320168 23094 connection.cc:535] server connection from 127.12.234.1:42961 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 04:44:58.320478 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 23315
I20260809 04:44:58.327620 23605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:58.321386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54598 (local address 127.12.234.2:33563)
0809 04:44:58.321905 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:58.321912 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:58.321937 (+ 25us) server_negotiation.cc:407] Connection header received
0809 04:44:58.321999 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:58.322005 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:58.322200 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:58.322340 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:58.322678 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:58.323656 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:58.324913 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:58.325423 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:58.325513 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:58.326391 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:58.326398 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:58.326410 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:58.326942 (+ 532us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:58.326996 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:58.327013 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:58.327098 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:58.327203 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 04:44:58.327303 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":242,"threads_started":1}
I20260809 04:44:58.328810 23445 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:58.405907 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 23315
I20260809 04:44:58.424353 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 23454
I20260809 04:44:58.432283 23607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:58.425345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45422 (local address 127.12.234.3:35945)
0809 04:44:58.426001 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:58.426014 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:58.426053 (+ 39us) server_negotiation.cc:407] Connection header received
0809 04:44:58.426113 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:58.426119 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:58.426322 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:58.426441 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:58.426802 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:58.427804 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:58.429213 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:44:58.429755 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:44:58.429851 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:44:58.430806 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:58.430814 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:58.430827 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:58.431457 (+ 630us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:58.431502 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:58.431521 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:58.431619 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:58.431765 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 04:44:58.431873 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":375,"threads_started":1}
I20260809 04:44:58.433439 23588 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:58.510588 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 23454
I20260809 04:44:58.529191 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 23079
I20260809 04:44:58.530454 23145 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:44:58.626149 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 23079
2026-08-09T04:44:58Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2403 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T04:44:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:44:58Z Disabled control of system clock
I20260809 04:44:58.706911 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:42283
--webserver_interface=127.12.234.62
--webserver_port=0
--builtin_ntp_servers=127.12.234.20:34615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:42283
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 04:44:59.018055 23613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:59.018322 23613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:59.018464 23613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:59.024945 23613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:44:59.025031 23613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:59.025087 23613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:59.025141 23613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:44:59.025177 23613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:44:59.033602 23613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:34615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:42283
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:42283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:59.034804 23613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:59.036423 23613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:59.046938 23618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:59.046824 23622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:59.046824 23619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:59.047394 23613 server_base.cc:1034] running on GCE node
I20260809 04:44:59.048221 23613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:59.049389 23613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:59.050590 23613 hybrid_clock.cc:648] HybridClock initialized: now 1786250699050550 us; error 34 us; skew 500 ppm
I20260809 04:44:59.050932 23613 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:44:59.052907 23613 webserver.cc:460] Webserver started at http://127.12.234.62:35097/ using document root <none> and password file <none>
I20260809 04:44:59.053546 23613 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:59.053661 23613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:59.053966 23613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:59.056737 23613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "03db222275484698be9a56b2ebcf03c6"
format_stamp: "Formatted at 2026-08-09 04:44:59 on dist-test-slave-1jjb"
server_key: "b8e19d84cfc57a780122e887acf22a77"
server_key_iv: "97b60e8de79bca79f88a2a6f11ef3676"
server_key_version: "encryptionkey@0"
I20260809 04:44:59.057457 23613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "03db222275484698be9a56b2ebcf03c6"
format_stamp: "Formatted at 2026-08-09 04:44:59 on dist-test-slave-1jjb"
server_key: "b8e19d84cfc57a780122e887acf22a77"
server_key_iv: "97b60e8de79bca79f88a2a6f11ef3676"
server_key_version: "encryptionkey@0"
I20260809 04:44:59.062188 23613 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:44:59.065152 23628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:59.066191 23613 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:44:59.066375 23613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "03db222275484698be9a56b2ebcf03c6"
format_stamp: "Formatted at 2026-08-09 04:44:59 on dist-test-slave-1jjb"
server_key: "b8e19d84cfc57a780122e887acf22a77"
server_key_iv: "97b60e8de79bca79f88a2a6f11ef3676"
server_key_version: "encryptionkey@0"
I20260809 04:44:59.066563 23613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:59.071856 23613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:59.072624 23613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:59.072908 23613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:59.090325 23613 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:42283
I20260809 04:44:59.090359 23684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:42283 every 8 connection(s)
I20260809 04:44:59.091620 23613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:44:59.095016 23685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:44:59.100013 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 23613
I20260809 04:44:59.100240 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:44:59.101154 13224 external_mini_cluster.cc:1428] Setting key 92cbb7aee5ef50522b08c2ad86d8005d
I20260809 04:44:59.106997 23685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03db222275484698be9a56b2ebcf03c6: Bootstrap starting.
I20260809 04:44:59.110397 23689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:59.103329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53006 (local address 127.12.234.62:42283)
0809 04:44:59.104471 (+ 1142us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:59.104484 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:59.104535 (+ 51us) server_negotiation.cc:407] Connection header received
0809 04:44:59.106625 (+ 2090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:59.106650 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:59.107126 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:59.107498 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:59.108455 (+ 957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:59.108464 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:59.108481 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:59.108506 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:59.108587 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:59.108614 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:59.108850 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:59.108984 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 04:44:59.109268 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":356,"threads_started":1}
I20260809 04:44:59.110868 23685 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03db222275484698be9a56b2ebcf03c6: Neither blocks nor log segments found. Creating new log.
I20260809 04:44:59.111984 23685 log.cc:824] T 00000000000000000000000000000000 P 03db222275484698be9a56b2ebcf03c6: Log is configured to *not* fsync() on all Append() calls
I20260809 04:44:59.114600 23685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03db222275484698be9a56b2ebcf03c6: No bootstrap required, opened a new log
I20260809 04:44:59.121614 23685 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03db222275484698be9a56b2ebcf03c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03db222275484698be9a56b2ebcf03c6" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42283 } }
I20260809 04:44:59.121901 23685 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03db222275484698be9a56b2ebcf03c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:44:59.121989 23685 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03db222275484698be9a56b2ebcf03c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03db222275484698be9a56b2ebcf03c6, State: Initialized, Role: FOLLOWER
I20260809 04:44:59.122429 23685 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03db222275484698be9a56b2ebcf03c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03db222275484698be9a56b2ebcf03c6" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42283 } }
I20260809 04:44:59.122650 23685 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03db222275484698be9a56b2ebcf03c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:44:59.122766 23685 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03db222275484698be9a56b2ebcf03c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:44:59.122884 23685 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03db222275484698be9a56b2ebcf03c6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:44:59.125167 23685 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03db222275484698be9a56b2ebcf03c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03db222275484698be9a56b2ebcf03c6" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42283 } }
I20260809 04:44:59.125622 23685 leader_election.cc:302] T 00000000000000000000000000000000 P 03db222275484698be9a56b2ebcf03c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03db222275484698be9a56b2ebcf03c6; no voters:
I20260809 04:44:59.126224 23685 leader_election.cc:288] T 00000000000000000000000000000000 P 03db222275484698be9a56b2ebcf03c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:44:59.126338 23691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03db222275484698be9a56b2ebcf03c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:44:59.127008 23691 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03db222275484698be9a56b2ebcf03c6 [term 1 LEADER]: Becoming Leader. State: Replica: 03db222275484698be9a56b2ebcf03c6, State: Running, Role: LEADER
I20260809 04:44:59.127552 23691 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03db222275484698be9a56b2ebcf03c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03db222275484698be9a56b2ebcf03c6" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42283 } }
I20260809 04:44:59.128026 23685 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03db222275484698be9a56b2ebcf03c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:44:59.130419 23693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03db222275484698be9a56b2ebcf03c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03db222275484698be9a56b2ebcf03c6. Latest consensus state: current_term: 1 leader_uuid: "03db222275484698be9a56b2ebcf03c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03db222275484698be9a56b2ebcf03c6" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42283 } } }
I20260809 04:44:59.130816 23693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03db222275484698be9a56b2ebcf03c6 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:59.131560 23692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03db222275484698be9a56b2ebcf03c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03db222275484698be9a56b2ebcf03c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03db222275484698be9a56b2ebcf03c6" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42283 } } }
I20260809 04:44:59.131939 23692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03db222275484698be9a56b2ebcf03c6 [sys.catalog]: This master's current role is: LEADER
I20260809 04:44:59.137292 23697 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:44:59.141855 23697 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:44:59.149721 23697 catalog_manager.cc:1347] Generated new cluster ID: 9bc4284bda264c1f840c68f486f47cc9
I20260809 04:44:59.149847 23697 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:44:59.166822 23697 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:44:59.167289 23697 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:44:59.173775 23697 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03db222275484698be9a56b2ebcf03c6: Generated new TSK 0
I20260809 04:44:59.174582 23697 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:44:59.183524 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:0
--local_ip_for_outbound_sockets=127.12.234.1
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:42283
--builtin_ntp_servers=127.12.234.20:34615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 04:44:59.499689 23710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:59.499964 23710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:59.500157 23710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:59.506459 23710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:59.506531 23710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:59.506637 23710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:44:59.514854 23710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:34615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:42283
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:59.516004 23710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:59.517695 23710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:59.529755 23715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:59.529664 23716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:59.529711 23718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:59.530447 23710 server_base.cc:1034] running on GCE node
I20260809 04:44:59.531021 23710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:59.531809 23710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:59.533030 23710 hybrid_clock.cc:648] HybridClock initialized: now 1786250699532954 us; error 70 us; skew 500 ppm
I20260809 04:44:59.533326 23710 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:44:59.535257 23710 webserver.cc:460] Webserver started at http://127.12.234.1:37589/ using document root <none> and password file <none>
I20260809 04:44:59.535867 23710 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:59.535946 23710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:59.536244 23710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:59.538789 23710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/data/instance:
uuid: "1e50f4a2393345cd9a6b10056060f931"
format_stamp: "Formatted at 2026-08-09 04:44:59 on dist-test-slave-1jjb"
server_key: "a58c4b4efb214a8d6894ba0fcf6bf51b"
server_key_iv: "7d8ce856d554a96b51b2279877a458c0"
server_key_version: "encryptionkey@0"
I20260809 04:44:59.539534 23710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance:
uuid: "1e50f4a2393345cd9a6b10056060f931"
format_stamp: "Formatted at 2026-08-09 04:44:59 on dist-test-slave-1jjb"
server_key: "a58c4b4efb214a8d6894ba0fcf6bf51b"
server_key_iv: "7d8ce856d554a96b51b2279877a458c0"
server_key_version: "encryptionkey@0"
I20260809 04:44:59.544202 23710 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.007s
I20260809 04:44:59.547327 23726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:59.548620 23710 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:44:59.548743 23710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "1e50f4a2393345cd9a6b10056060f931"
format_stamp: "Formatted at 2026-08-09 04:44:59 on dist-test-slave-1jjb"
server_key: "a58c4b4efb214a8d6894ba0fcf6bf51b"
server_key_iv: "7d8ce856d554a96b51b2279877a458c0"
server_key_version: "encryptionkey@0"
I20260809 04:44:59.548899 23710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:59.553902 23710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:59.554569 23710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:59.554780 23710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:59.555570 23710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:59.556804 23710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:59.556878 23710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:59.556947 23710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:59.556994 23710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:59.588791 23710 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:40557
I20260809 04:44:59.588840 23845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:40557 every 8 connection(s)
I20260809 04:44:59.590183 23710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:44:59.598115 23689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:59.592667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:56595 (local address 127.12.234.62:42283)
0809 04:44:59.592891 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 04:44:59.592896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:44:59.594101 (+ 1205us) server_negotiation.cc:407] Connection header received
0809 04:44:59.595732 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:44:59.595738 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:44:59.595823 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:44:59.595947 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:44:59.597249 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:44:59.597255 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:44:59.597260 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:44:59.597303 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:44:59.597347 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:44:59.597361 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:44:59.597512 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 04:44:59.597736 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 04:44:59.597822 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 04:44:59.598951 23848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:44:59.593103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:42283 (local address 127.12.234.1:56595)
0809 04:44:59.593935 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0809 04:44:59.593975 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 04:44:59.595465 (+ 1490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:44:59.596227 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:44:59.596234 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:44:59.596320 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:44:59.596904 (+ 584us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:44:59.596906 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:44:59.596957 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:44:59.597044 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:44:59.597474 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:44:59.597488 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:44:59.597498 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 04:44:59.597818 (+ 320us) client_negotiation.cc:240] Negotiation successful
0809 04:44:59.598042 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":265,"threads_started":1}
I20260809 04:44:59.600283 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 23710
I20260809 04:44:59.600522 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance
I20260809 04:44:59.600523 23846 heartbeater.cc:347] Connected to a master server at 127.12.234.62:42283
I20260809 04:44:59.600823 23846 heartbeater.cc:464] Registering TS with master...
I20260809 04:44:59.601608 13224 external_mini_cluster.cc:1428] Setting key 8fa66164d10b60a742be9025e541df31
I20260809 04:44:59.601614 23846 heartbeater.cc:511] Master 127.12.234.62:42283 requested a full tablet report, sending...
I20260809 04:44:59.603899 23648 ts_manager.cc:194] Registered new tserver with Master: 1e50f4a2393345cd9a6b10056060f931 (127.12.234.1:40557)
I20260809 04:44:59.604593 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:0
--local_ip_for_outbound_sockets=127.12.234.2
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:42283
--builtin_ntp_servers=127.12.234.20:34615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 04:44:59.913529 23849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:44:59.913762 23849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:44:59.913839 23849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:44:59.920229 23849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:44:59.920313 23849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:44:59.920477 23849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:44:59.928637 23849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:34615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:42283
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:44:59.929800 23849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:44:59.931447 23849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:44:59.945089 23855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:59.945128 23857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:44:59.945283 23859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:44:59.946125 23849 server_base.cc:1034] running on GCE node
I20260809 04:44:59.946631 23849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:44:59.947468 23849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:44:59.948683 23849 hybrid_clock.cc:648] HybridClock initialized: now 1786250699948642 us; error 46 us; skew 500 ppm
I20260809 04:44:59.948967 23849 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:44:59.951303 23849 webserver.cc:460] Webserver started at http://127.12.234.2:44731/ using document root <none> and password file <none>
I20260809 04:44:59.951947 23849 fs_manager.cc:359] Metadata directory not provided
I20260809 04:44:59.952066 23849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:44:59.952405 23849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:44:59.954957 23849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/data/instance:
uuid: "1b2dfa8b4e2e4519950548d05757abc5"
format_stamp: "Formatted at 2026-08-09 04:44:59 on dist-test-slave-1jjb"
server_key: "b050ec8ee3271a18faf391e9d8b94f4f"
server_key_iv: "b6cfeeabe690f75b202f94e702a18a07"
server_key_version: "encryptionkey@0"
I20260809 04:44:59.955651 23849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance:
uuid: "1b2dfa8b4e2e4519950548d05757abc5"
format_stamp: "Formatted at 2026-08-09 04:44:59 on dist-test-slave-1jjb"
server_key: "b050ec8ee3271a18faf391e9d8b94f4f"
server_key_iv: "b6cfeeabe690f75b202f94e702a18a07"
server_key_version: "encryptionkey@0"
I20260809 04:44:59.960191 23849 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260809 04:44:59.963181 23865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:59.964401 23849 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.003s
I20260809 04:44:59.964586 23849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "1b2dfa8b4e2e4519950548d05757abc5"
format_stamp: "Formatted at 2026-08-09 04:44:59 on dist-test-slave-1jjb"
server_key: "b050ec8ee3271a18faf391e9d8b94f4f"
server_key_iv: "b6cfeeabe690f75b202f94e702a18a07"
server_key_version: "encryptionkey@0"
I20260809 04:44:59.964824 23849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:44:59.969911 23849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:44:59.970750 23849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:44:59.971015 23849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:44:59.971881 23849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:44:59.973178 23849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:44:59.973287 23849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:44:59.973404 23849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:44:59.973481 23849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:00.005311 23849 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:33777
I20260809 04:45:00.005401 23988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:33777 every 8 connection(s)
I20260809 04:45:00.006528 23849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:45:00.011439 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 23849
I20260809 04:45:00.011633 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance
I20260809 04:45:00.012177 13224 external_mini_cluster.cc:1428] Setting key 9a7ac6a4c90d3032d0d9bbc3f2936565
I20260809 04:45:00.015264 23689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:00.009197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:32799 (local address 127.12.234.62:42283)
0809 04:45:00.009429 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:00.009436 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:00.010787 (+ 1351us) server_negotiation.cc:407] Connection header received
0809 04:45:00.013055 (+ 2268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:00.013061 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:00.013134 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:00.013244 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:00.014574 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:00.014579 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:00.014582 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:00.014599 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:00.014632 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:00.014661 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:00.014775 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:00.014988 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 04:45:00.015040 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 04:45:00.015280 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:0
--local_ip_for_outbound_sockets=127.12.234.3
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:42283
--builtin_ntp_servers=127.12.234.20:34615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 04:45:00.016477 23991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:00.009762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:42283 (local address 127.12.234.2:32799)
0809 04:45:00.010633 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:00.010668 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:00.012803 (+ 2135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:00.013467 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:00.013475 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:00.013558 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:00.014253 (+ 695us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:45:00.014256 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:00.014300 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:45:00.014405 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:00.014769 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:00.014779 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:00.014786 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 04:45:00.015072 (+ 286us) client_negotiation.cc:240] Negotiation successful
0809 04:45:00.015318 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":281,"threads_started":1}
I20260809 04:45:00.017793 23989 heartbeater.cc:347] Connected to a master server at 127.12.234.62:42283
I20260809 04:45:00.018097 23989 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:00.018699 23989 heartbeater.cc:511] Master 127.12.234.62:42283 requested a full tablet report, sending...
I20260809 04:45:00.020144 23648 ts_manager.cc:194] Registered new tserver with Master: 1b2dfa8b4e2e4519950548d05757abc5 (127.12.234.2:33777)
W20260809 04:45:00.326653 23992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:00.326996 23992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:00.327143 23992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:00.334193 23992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:00.334283 23992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:00.334400 23992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:45:00.342980 23992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:34615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:42283
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:45:00.344084 23992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:00.345652 23992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:00.357986 23998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:00.357940 23999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:00.358807 24001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:00.358809 23992 server_base.cc:1034] running on GCE node
I20260809 04:45:00.359488 23992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:00.360476 23992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:00.361697 23992 hybrid_clock.cc:648] HybridClock initialized: now 1786250700361654 us; error 57 us; skew 500 ppm
I20260809 04:45:00.361997 23992 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:45:00.363924 23992 webserver.cc:460] Webserver started at http://127.12.234.3:35259/ using document root <none> and password file <none>
I20260809 04:45:00.364580 23992 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:00.364675 23992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:00.364943 23992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:00.367550 23992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/data/instance:
uuid: "9cdb9e9b79c0495983b6f2b0c3e3d450"
format_stamp: "Formatted at 2026-08-09 04:45:00 on dist-test-slave-1jjb"
server_key: "a2b674a021b3f52a7d2515fbb2a0a6b5"
server_key_iv: "780ffa1e43f69d751c7cd0ab872b0718"
server_key_version: "encryptionkey@0"
I20260809 04:45:00.368353 23992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance:
uuid: "9cdb9e9b79c0495983b6f2b0c3e3d450"
format_stamp: "Formatted at 2026-08-09 04:45:00 on dist-test-slave-1jjb"
server_key: "a2b674a021b3f52a7d2515fbb2a0a6b5"
server_key_iv: "780ffa1e43f69d751c7cd0ab872b0718"
server_key_version: "encryptionkey@0"
I20260809 04:45:00.373525 23992 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260809 04:45:00.376806 24007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:00.377911 23992 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:45:00.378057 23992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "9cdb9e9b79c0495983b6f2b0c3e3d450"
format_stamp: "Formatted at 2026-08-09 04:45:00 on dist-test-slave-1jjb"
server_key: "a2b674a021b3f52a7d2515fbb2a0a6b5"
server_key_iv: "780ffa1e43f69d751c7cd0ab872b0718"
server_key_version: "encryptionkey@0"
I20260809 04:45:00.378247 23992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:00.383400 23992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:00.384222 23992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:00.384479 23992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:00.385262 23992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:00.386500 23992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:00.386605 23992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:00.386719 23992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:00.386797 23992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:00.418431 23992 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:41491
I20260809 04:45:00.418509 24128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:41491 every 8 connection(s)
I20260809 04:45:00.419651 23992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:45:00.422919 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 23992
I20260809 04:45:00.423107 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance
I20260809 04:45:00.423492 13224 external_mini_cluster.cc:1428] Setting key 889c5e8a0b99df00570f3fd1988a8c9f
I20260809 04:45:00.427531 23689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:00.422181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:45321 (local address 127.12.234.62:42283)
0809 04:45:00.422354 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:00.422358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:00.423672 (+ 1314us) server_negotiation.cc:407] Connection header received
0809 04:45:00.425583 (+ 1911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:00.425588 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:00.425667 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:00.425771 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:00.426956 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:00.426960 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:00.426963 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:00.426977 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:00.427008 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:00.427018 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:00.427100 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:00.427285 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 04:45:00.427328 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:45:00.428483 24131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:00.422542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:42283 (local address 127.12.234.3:45321)
0809 04:45:00.423502 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:00.423559 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:00.425271 (+ 1712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:00.425961 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:00.425967 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:00.426045 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:00.426707 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:45:00.426710 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:00.426744 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:45:00.426818 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:00.427098 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:00.427105 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:00.427110 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 04:45:00.427325 (+ 215us) client_negotiation.cc:240] Negotiation successful
0809 04:45:00.427492 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":346,"threads_started":1}
I20260809 04:45:00.429765 24129 heartbeater.cc:347] Connected to a master server at 127.12.234.62:42283
I20260809 04:45:00.430053 24129 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:00.430584 24129 heartbeater.cc:511] Master 127.12.234.62:42283 requested a full tablet report, sending...
I20260809 04:45:00.431797 23648 ts_manager.cc:194] Registered new tserver with Master: 9cdb9e9b79c0495983b6f2b0c3e3d450 (127.12.234.3:41491)
I20260809 04:45:00.438505 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 04:45:00.444375 23689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:45:00.442968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53016 (local address 127.12.234.62:42283)
0809 04:45:00.443118 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:00.443122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:00.443599 (+ 477us) server_negotiation.cc:407] Connection header received
0809 04:45:00.443712 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:00.443716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:00.443759 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:00.443843 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:00.444150 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53016: BlockingRecv error: recv got EOF from 127.0.0.1:53016 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 04:45:00.445389 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 23710
I20260809 04:45:00.448711 24139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:00.446232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58788 (local address 127.12.234.1:40557)
0809 04:45:00.446668 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:00.446674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:00.446694 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:45:00.446743 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:00.446748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:00.446901 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:00.447043 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:00.447937 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:00.447942 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:00.447951 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:00.448007 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:00.448049 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:00.448063 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:00.448232 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:00.448348 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 04:45:00.448445 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":193,"threads_started":1}
I20260809 04:45:00.449867 23841 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:45:00.527299 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 23710
I20260809 04:45:00.547142 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 23849
I20260809 04:45:00.550292 24141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:00.547898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36390 (local address 127.12.234.2:33777)
0809 04:45:00.548413 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:00.548427 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:00.548452 (+ 25us) server_negotiation.cc:407] Connection header received
0809 04:45:00.548511 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:00.548517 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:00.548721 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:00.548841 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:00.549686 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:00.549693 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:00.549703 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:00.549714 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:00.549752 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:00.549764 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:00.549873 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:00.549979 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 04:45:00.550071 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":251,"threads_started":1}
I20260809 04:45:00.551424 23984 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:45:00.628684 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 23849
I20260809 04:45:00.647969 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 23992
I20260809 04:45:00.651953 24143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:00.649076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45914 (local address 127.12.234.3:41491)
0809 04:45:00.649812 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:00.649819 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:00.649849 (+ 30us) server_negotiation.cc:407] Connection header received
0809 04:45:00.649918 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:00.649924 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:00.650143 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:00.650262 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:00.651282 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:00.651287 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:00.651297 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:45:00.651307 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:00.651344 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:45:00.651356 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:00.651470 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:00.651565 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 04:45:00.651679 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":413,"threads_started":1}
I20260809 04:45:00.653247 24124 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:45:00.730854 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 23992
I20260809 04:45:00.751369 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 23613
I20260809 04:45:00.752596 23680 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:45:00.849264 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 23613
2026-08-09T04:45:00Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2223 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:45:00 dist-test-slave-1jjb krb5kdc[24148](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:45:00 dist-test-slave-1jjb krb5kdc[24148](info): set up 2 sockets
Aug 09 04:45:00 dist-test-slave-1jjb krb5kdc[24148](info): commencing operation
krb5kdc: starting...
W20260809 04:45:01.953440 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.004s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:45:02 dist-test-slave-1jjb krb5kdc[24148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:45:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:45:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:45:02.194833 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:33199
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:43313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:33199
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:02.500370 24161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:02.500636 24161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:02.500731 24161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:02.507133 24161 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:45:02.507228 24161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:02.507278 24161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:02.507341 24161 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:45:02.507390 24161 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:45:02.516220 24161 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:43313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:33199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:33199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:45:02.517285 24161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:02.518390 24161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:02.529547 24166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:02.529577 24169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:02.529577 24167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:02.530404 24161 server_base.cc:1034] running on GCE node
I20260809 04:45:02.531203 24161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:02.532527 24161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:02.533776 24161 hybrid_clock.cc:648] HybridClock initialized: now 1786250702533723 us; error 34 us; skew 500 ppm
I20260809 04:45:02.534113 24161 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:45:02 dist-test-slave-1jjb krb5kdc[24148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250702, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:02.542270 24161 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:45:02.543704 24161 webserver.cc:460] Webserver started at http://127.12.234.62:34575/ using document root <none> and password file <none>
I20260809 04:45:02.544373 24161 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:02.544507 24161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:02.544785 24161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:02.547453 24161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "afe2ada7846f419299fd145403c5dc66"
format_stamp: "Formatted at 2026-08-09 04:45:02 on dist-test-slave-1jjb"
server_key: "2daeedf7c8dc0da6d893fdead3489334"
server_key_iv: "5728bf1a1385f1b4f537b7be4fd0ec7e"
server_key_version: "encryptionkey@0"
I20260809 04:45:02.548170 24161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "afe2ada7846f419299fd145403c5dc66"
format_stamp: "Formatted at 2026-08-09 04:45:02 on dist-test-slave-1jjb"
server_key: "2daeedf7c8dc0da6d893fdead3489334"
server_key_iv: "5728bf1a1385f1b4f537b7be4fd0ec7e"
server_key_version: "encryptionkey@0"
I20260809 04:45:02.552881 24161 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:45:02.555814 24177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:02.556912 24161 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:45:02.557075 24161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "afe2ada7846f419299fd145403c5dc66"
format_stamp: "Formatted at 2026-08-09 04:45:02 on dist-test-slave-1jjb"
server_key: "2daeedf7c8dc0da6d893fdead3489334"
server_key_iv: "5728bf1a1385f1b4f537b7be4fd0ec7e"
server_key_version: "encryptionkey@0"
I20260809 04:45:02.557241 24161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:02.569634 24161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:02.573122 24161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:02.573383 24161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:02.590845 24161 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:33199
I20260809 04:45:02.590889 24235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:33199 every 8 connection(s)
I20260809 04:45:02.592104 24161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:45:02.595371 24236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:02.596676 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 24161
I20260809 04:45:02.596928 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:45:02.597617 13224 external_mini_cluster.cc:1428] Setting key 0784c7dde2f6278cf2b9d7c0f962b91e
I20260809 04:45:02.606979 24236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afe2ada7846f419299fd145403c5dc66: Bootstrap starting.
I20260809 04:45:02.610649 24236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afe2ada7846f419299fd145403c5dc66: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:02.611950 24236 log.cc:824] T 00000000000000000000000000000000 P afe2ada7846f419299fd145403c5dc66: Log is configured to *not* fsync() on all Append() calls
Aug 09 04:45:02 dist-test-slave-1jjb krb5kdc[24148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:45:02.614764 24236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afe2ada7846f419299fd145403c5dc66: No bootstrap required, opened a new log
I20260809 04:45:02.619899 24239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:02.599628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36744 (local address 127.12.234.62:33199)
0809 04:45:02.600503 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:02.600514 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:02.600556 (+ 42us) server_negotiation.cc:407] Connection header received
0809 04:45:02.602257 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:02.602281 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:02.602651 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:02.602952 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:02.604794 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.606058 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.607783 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:02.608549 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:02.608699 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:02.613659 (+ 4960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:02.613667 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:02.613680 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:02.613712 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:02.616477 (+ 2765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:02.617097 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:02.617111 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:02.617116 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:02.617273 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:02.617627 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:02.617632 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:02.617635 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:02.618062 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:02.618423 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:02.618758 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 04:45:02.619090 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":436,"threads_started":1}
I20260809 04:45:02.621876 24236 raft_consensus.cc:348] T 00000000000000000000000000000000 P afe2ada7846f419299fd145403c5dc66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afe2ada7846f419299fd145403c5dc66" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 33199 } }
I20260809 04:45:02.622201 24236 raft_consensus.cc:374] T 00000000000000000000000000000000 P afe2ada7846f419299fd145403c5dc66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:02.622319 24236 raft_consensus.cc:729] T 00000000000000000000000000000000 P afe2ada7846f419299fd145403c5dc66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afe2ada7846f419299fd145403c5dc66, State: Initialized, Role: FOLLOWER
I20260809 04:45:02.622853 24236 consensus_queue.cc:260] T 00000000000000000000000000000000 P afe2ada7846f419299fd145403c5dc66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afe2ada7846f419299fd145403c5dc66" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 33199 } }
I20260809 04:45:02.623057 24236 raft_consensus.cc:388] T 00000000000000000000000000000000 P afe2ada7846f419299fd145403c5dc66 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:45:02.623200 24236 raft_consensus.cc:482] T 00000000000000000000000000000000 P afe2ada7846f419299fd145403c5dc66 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:45:02.623350 24236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afe2ada7846f419299fd145403c5dc66 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:02.625618 24236 raft_consensus.cc:504] T 00000000000000000000000000000000 P afe2ada7846f419299fd145403c5dc66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afe2ada7846f419299fd145403c5dc66" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 33199 } }
I20260809 04:45:02.626055 24236 leader_election.cc:302] T 00000000000000000000000000000000 P afe2ada7846f419299fd145403c5dc66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afe2ada7846f419299fd145403c5dc66; no voters:
I20260809 04:45:02.626526 24236 leader_election.cc:288] T 00000000000000000000000000000000 P afe2ada7846f419299fd145403c5dc66 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:45:02.626633 24241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afe2ada7846f419299fd145403c5dc66 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:02.627311 24241 raft_consensus.cc:686] T 00000000000000000000000000000000 P afe2ada7846f419299fd145403c5dc66 [term 1 LEADER]: Becoming Leader. State: Replica: afe2ada7846f419299fd145403c5dc66, State: Running, Role: LEADER
I20260809 04:45:02.627784 24241 consensus_queue.cc:237] T 00000000000000000000000000000000 P afe2ada7846f419299fd145403c5dc66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afe2ada7846f419299fd145403c5dc66" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 33199 } }
I20260809 04:45:02.628399 24236 sys_catalog.cc:564] T 00000000000000000000000000000000 P afe2ada7846f419299fd145403c5dc66 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:45:02.630342 24243 sys_catalog.cc:455] T 00000000000000000000000000000000 P afe2ada7846f419299fd145403c5dc66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afe2ada7846f419299fd145403c5dc66. Latest consensus state: current_term: 1 leader_uuid: "afe2ada7846f419299fd145403c5dc66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afe2ada7846f419299fd145403c5dc66" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 33199 } } }
I20260809 04:45:02.630761 24243 sys_catalog.cc:458] T 00000000000000000000000000000000 P afe2ada7846f419299fd145403c5dc66 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:02.630959 24242 sys_catalog.cc:455] T 00000000000000000000000000000000 P afe2ada7846f419299fd145403c5dc66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afe2ada7846f419299fd145403c5dc66" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afe2ada7846f419299fd145403c5dc66" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 33199 } } }
I20260809 04:45:02.631495 24242 sys_catalog.cc:458] T 00000000000000000000000000000000 P afe2ada7846f419299fd145403c5dc66 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:02.633152 24248 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:45:02.637945 24248 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:45:02.646489 24248 catalog_manager.cc:1347] Generated new cluster ID: bdb68387eaf24ede8497a2c1877fddaf
I20260809 04:45:02.646595 24248 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:45:02.656761 24248 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:45:02.657941 24248 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:45:02.673657 24248 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afe2ada7846f419299fd145403c5dc66: Generated new TSK 0
I20260809 04:45:02.674417 24248 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
I20260809 04:45:02.749061 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:0
--local_ip_for_outbound_sockets=127.12.234.1
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:33199
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:43313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:03.068814 24263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:03.069097 24263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:03.069264 24263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:03.076679 24263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:03.076821 24263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:03.076973 24263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:45:03.085983 24263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:43313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:33199
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:45:03.087167 24263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:03.088517 24263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:03.105070 24269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:03.109838 24268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:03.110489 24273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:03.111171 24263 server_base.cc:1034] running on GCE node
I20260809 04:45:03.111804 24263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:03.112805 24263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:03.114063 24263 hybrid_clock.cc:648] HybridClock initialized: now 1786250703114011 us; error 53 us; skew 500 ppm
I20260809 04:45:03.114369 24263 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:45:03 dist-test-slave-1jjb krb5kdc[24148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250703, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:03.117449 24263 init.cc:377] Logged in from keytab as kudu/127.12.234.1@KRBTEST.COM (short username kudu)
I20260809 04:45:03.119177 24263 webserver.cc:460] Webserver started at http://127.12.234.1:33385/ using document root <none> and password file <none>
I20260809 04:45:03.119850 24263 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:03.119992 24263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:03.120332 24263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:03.123283 24263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/data/instance:
uuid: "d703b295100f4dd5848903cf5fb5d3bd"
format_stamp: "Formatted at 2026-08-09 04:45:03 on dist-test-slave-1jjb"
server_key: "dae9702124262053704592c842740545"
server_key_iv: "70ce1a58c60e0ffd3f413eb593a26867"
server_key_version: "encryptionkey@0"
I20260809 04:45:03.124027 24263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance:
uuid: "d703b295100f4dd5848903cf5fb5d3bd"
format_stamp: "Formatted at 2026-08-09 04:45:03 on dist-test-slave-1jjb"
server_key: "dae9702124262053704592c842740545"
server_key_iv: "70ce1a58c60e0ffd3f413eb593a26867"
server_key_version: "encryptionkey@0"
I20260809 04:45:03.129262 24263 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 04:45:03.136579 24280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:03.137862 24263 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.004s
I20260809 04:45:03.138015 24263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "d703b295100f4dd5848903cf5fb5d3bd"
format_stamp: "Formatted at 2026-08-09 04:45:03 on dist-test-slave-1jjb"
server_key: "dae9702124262053704592c842740545"
server_key_iv: "70ce1a58c60e0ffd3f413eb593a26867"
server_key_version: "encryptionkey@0"
I20260809 04:45:03.138196 24263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:03.161410 24263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:03.166287 24263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:03.166543 24263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:03.167608 24263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:03.169267 24263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:03.169378 24263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:03.169533 24263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:03.169610 24263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:03.209059 24263 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:36017
I20260809 04:45:03.209111 24409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:36017 every 8 connection(s)
I20260809 04:45:03.210296 24263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:45:03.212138 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 24263
I20260809 04:45:03.212355 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance
I20260809 04:45:03.213037 13224 external_mini_cluster.cc:1428] Setting key f0c35a0b0e0c0a795a6fb8e2685e2f6f
Aug 09 04:45:03 dist-test-slave-1jjb krb5kdc[24148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250703, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:45:03.248162 24412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:03.213514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:58389 (local address 127.12.234.62:33199)
0809 04:45:03.228766 (+ 15252us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:03.228774 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:03.232128 (+ 3354us) server_negotiation.cc:407] Connection header received
0809 04:45:03.233832 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:03.233837 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:03.233918 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:03.234048 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:03.235695 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:03.236491 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:03.237860 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:03.238275 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:03.238401 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:03.242288 (+ 3887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:03.242293 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:03.242296 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:03.242331 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:03.244984 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:03.245884 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:03.245888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:03.245890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:03.246013 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:03.246498 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:03.246501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:03.246503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:03.246731 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:03.246835 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:03.247671 (+ 836us) server_negotiation.cc:299] Negotiation successful
0809 04:45:03.247824 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15093,"thread_start_us":15248,"threads_started":1}
WARNING: no policy specified for kudu/127.12.234.2@KRBTEST.COM; defaulting to no policy
I20260809 04:45:03.249497 24413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:03.214070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:33199 (local address 127.12.234.1:58389)
0809 04:45:03.231649 (+ 17579us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:03.231707 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:03.233522 (+ 1815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:03.234325 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:03.234333 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:03.234867 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:03.235542 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:03.235558 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:03.236639 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:03.236644 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:03.237692 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:03.237703 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:03.238515 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:03.238522 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:03.238780 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:03.239536 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:03.239558 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:03.242070 (+ 2512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:03.245111 (+ 3041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:03.245119 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:03.245133 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:03.245732 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:03.246138 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:03.246144 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:03.246151 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:03.246391 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:03.246866 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:03.246874 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:03.247262 (+ 388us) client_negotiation.cc:769] Sending connection context
0809 04:45:03.247588 (+ 326us) client_negotiation.cc:240] Negotiation successful
0809 04:45:03.248131 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17294,"thread_start_us":17051,"threads_started":1}
I20260809 04:45:03.250957 24410 heartbeater.cc:347] Connected to a master server at 127.12.234.62:33199
I20260809 04:45:03.251255 24410 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.2@KRBTEST.COM" created.
I20260809 04:45:03.252197 24410 heartbeater.cc:511] Master 127.12.234.62:33199 requested a full tablet report, sending...
I20260809 04:45:03.254136 24197 ts_manager.cc:194] Registered new tserver with Master: d703b295100f4dd5848903cf5fb5d3bd (127.12.234.1:36017)
I20260809 04:45:03.255420 24197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.1@KRBTEST.COM'} at 127.12.234.1:58389
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
I20260809 04:45:03.281610 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:0
--local_ip_for_outbound_sockets=127.12.234.2
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:33199
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:43313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:03.604931 24416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:03.605247 24416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:03.605382 24416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:03.614899 24416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:03.614986 24416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:03.615108 24416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:45:03.623759 24416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:43313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:33199
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:45:03.624898 24416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:03.626042 24416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:03.638113 24424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:03.638154 24421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:03.638222 24422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:03.639219 24416 server_base.cc:1034] running on GCE node
I20260809 04:45:03.639806 24416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:03.640780 24416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:03.641988 24416 hybrid_clock.cc:648] HybridClock initialized: now 1786250703641934 us; error 46 us; skew 500 ppm
I20260809 04:45:03.642284 24416 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:45:03 dist-test-slave-1jjb krb5kdc[24148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250703, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:03.645351 24416 init.cc:377] Logged in from keytab as kudu/127.12.234.2@KRBTEST.COM (short username kudu)
I20260809 04:45:03.646735 24416 webserver.cc:460] Webserver started at http://127.12.234.2:35497/ using document root <none> and password file <none>
I20260809 04:45:03.647323 24416 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:03.647425 24416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:03.647678 24416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:03.650441 24416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/data/instance:
uuid: "fe443e30ed2b4349b99d5dcbe20e8640"
format_stamp: "Formatted at 2026-08-09 04:45:03 on dist-test-slave-1jjb"
server_key: "a266a330ba4df49149d3dd9b49294f25"
server_key_iv: "ebbf402a19bf6fac4cf63d893c683fcf"
server_key_version: "encryptionkey@0"
I20260809 04:45:03.651117 24416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance:
uuid: "fe443e30ed2b4349b99d5dcbe20e8640"
format_stamp: "Formatted at 2026-08-09 04:45:03 on dist-test-slave-1jjb"
server_key: "a266a330ba4df49149d3dd9b49294f25"
server_key_iv: "ebbf402a19bf6fac4cf63d893c683fcf"
server_key_version: "encryptionkey@0"
I20260809 04:45:03.655937 24416 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260809 04:45:03.659137 24431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:03.660612 24416 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:45:03.660784 24416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "fe443e30ed2b4349b99d5dcbe20e8640"
format_stamp: "Formatted at 2026-08-09 04:45:03 on dist-test-slave-1jjb"
server_key: "a266a330ba4df49149d3dd9b49294f25"
server_key_iv: "ebbf402a19bf6fac4cf63d893c683fcf"
server_key_version: "encryptionkey@0"
I20260809 04:45:03.660985 24416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:03.677606 24416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:03.682025 24416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:03.682341 24416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:03.683260 24416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:03.684757 24416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:03.684864 24416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:03.685019 24416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:03.685137 24416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:03.716882 24416 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:42731
I20260809 04:45:03.716964 24553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:42731 every 8 connection(s)
I20260809 04:45:03.718067 24416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:45:03.727412 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 24416
I20260809 04:45:03.727633 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance
I20260809 04:45:03.728065 13224 external_mini_cluster.cc:1428] Setting key 884c891a9067debb63f9f7b16303650f
Aug 09 04:45:03 dist-test-slave-1jjb krb5kdc[24148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250703, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:45:03.737974 24412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:03.720466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:56945 (local address 127.12.234.62:33199)
0809 04:45:03.720638 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:03.720643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:03.721753 (+ 1110us) server_negotiation.cc:407] Connection header received
0809 04:45:03.723366 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:03.723370 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:03.723434 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:03.723520 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:03.724896 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:03.725555 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:03.726882 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:03.727436 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:03.727570 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:03.731454 (+ 3884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:03.731463 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:03.731471 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:03.731516 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:03.735027 (+ 3511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:03.735679 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:03.735686 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:03.735691 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:03.735827 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:03.736248 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:03.736254 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:03.736259 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:03.736600 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:03.736765 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:03.737339 (+ 574us) server_negotiation.cc:299] Negotiation successful
0809 04:45:03.737569 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:45:03.738839 24556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:03.720928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:33199 (local address 127.12.234.2:56945)
0809 04:45:03.721609 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:03.721644 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:03.723135 (+ 1491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:03.723706 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:03.723713 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:03.724199 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:03.724757 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:03.724775 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:03.725676 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:03.725680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:03.726767 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:03.726778 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:03.727686 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:03.727696 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:03.727930 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:03.728714 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:03.728736 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:03.731219 (+ 2483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:03.735155 (+ 3936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:03.735160 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:03.735170 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:03.735581 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:03.735920 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:03.735923 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:03.735925 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:03.736152 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:03.736735 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:03.736740 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:03.737016 (+ 276us) client_negotiation.cc:769] Sending connection context
0809 04:45:03.737212 (+ 196us) client_negotiation.cc:240] Negotiation successful
0809 04:45:03.737624 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":251,"threads_started":1}
I20260809 04:45:03.740635 24554 heartbeater.cc:347] Connected to a master server at 127.12.234.62:33199
I20260809 04:45:03.740980 24554 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:03.741889 24554 heartbeater.cc:511] Master 127.12.234.62:33199 requested a full tablet report, sending...
I20260809 04:45:03.743505 24197 ts_manager.cc:194] Registered new tserver with Master: fe443e30ed2b4349b99d5dcbe20e8640 (127.12.234.2:42731)
I20260809 04:45:03.744458 24197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.2@KRBTEST.COM'} at 127.12.234.2:56945
WARNING: no policy specified for kudu/127.12.234.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
I20260809 04:45:03.789219 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:0
--local_ip_for_outbound_sockets=127.12.234.3
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:33199
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:43313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:04.111279 24559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:04.111542 24559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:04.111630 24559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:04.118400 24559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:04.118487 24559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:04.118590 24559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:45:04.127624 24559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:43313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:33199
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:45:04.128744 24559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:04.129959 24559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:04.143100 24564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:04.143105 24565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:04.143877 24568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:04.144269 24559 server_base.cc:1034] running on GCE node
I20260809 04:45:04.144846 24559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:04.145699 24559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:04.146929 24559 hybrid_clock.cc:648] HybridClock initialized: now 1786250704146874 us; error 64 us; skew 500 ppm
I20260809 04:45:04.147223 24559 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:45:04 dist-test-slave-1jjb krb5kdc[24148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250704, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:04.150337 24559 init.cc:377] Logged in from keytab as kudu/127.12.234.3@KRBTEST.COM (short username kudu)
I20260809 04:45:04.151629 24559 webserver.cc:460] Webserver started at http://127.12.234.3:38933/ using document root <none> and password file <none>
I20260809 04:45:04.152318 24559 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:04.152415 24559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:04.152647 24559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:45:04.155257 24559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/data/instance:
uuid: "65fdfbd0b0594136b96ad5e597f757b9"
format_stamp: "Formatted at 2026-08-09 04:45:04 on dist-test-slave-1jjb"
server_key: "f76cec50da5a63237c8187bdb3a78efa"
server_key_iv: "36238f364eb0274bae84b230c919caae"
server_key_version: "encryptionkey@0"
I20260809 04:45:04.155910 24559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance:
uuid: "65fdfbd0b0594136b96ad5e597f757b9"
format_stamp: "Formatted at 2026-08-09 04:45:04 on dist-test-slave-1jjb"
server_key: "f76cec50da5a63237c8187bdb3a78efa"
server_key_iv: "36238f364eb0274bae84b230c919caae"
server_key_version: "encryptionkey@0"
I20260809 04:45:04.160682 24559 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 04:45:04.163908 24575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:04.165154 24559 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:45:04.165313 24559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "65fdfbd0b0594136b96ad5e597f757b9"
format_stamp: "Formatted at 2026-08-09 04:45:04 on dist-test-slave-1jjb"
server_key: "f76cec50da5a63237c8187bdb3a78efa"
server_key_iv: "36238f364eb0274bae84b230c919caae"
server_key_version: "encryptionkey@0"
I20260809 04:45:04.165463 24559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:04.180223 24559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:04.183784 24559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:04.184095 24559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:04.184984 24559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:04.186230 24559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:04.186298 24559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:04.186417 24559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:04.186465 24559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:04.222510 24559 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:34431
I20260809 04:45:04.222530 24704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:34431 every 8 connection(s)
I20260809 04:45:04.223687 24559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:45:04.233001 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 24559
I20260809 04:45:04.233204 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance
I20260809 04:45:04.233747 13224 external_mini_cluster.cc:1428] Setting key dd46c67af070490956abad97998da4d0
Aug 09 04:45:04 dist-test-slave-1jjb krb5kdc[24148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250704, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:45:04.243948 24412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.226140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:40063 (local address 127.12.234.62:33199)
0809 04:45:04.226310 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:04.226314 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:04.227389 (+ 1075us) server_negotiation.cc:407] Connection header received
0809 04:45:04.229690 (+ 2301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:04.229700 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:04.229788 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:04.229922 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:04.231335 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.232163 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.234300 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.234969 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.235101 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.238656 (+ 3555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:04.238660 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:04.238663 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:04.238692 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:04.241407 (+ 2715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:04.242010 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:04.242013 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:04.242015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:04.242158 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:04.242492 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:04.242495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:04.242496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:04.242755 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:04.242904 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:04.243530 (+ 626us) server_negotiation.cc:299] Negotiation successful
0809 04:45:04.243702 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:45:04.244771 24708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.226532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:33199 (local address 127.12.234.3:40063)
0809 04:45:04.227248 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:04.227286 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:04.229348 (+ 2062us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:04.230119 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:04.230127 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:04.230633 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:04.231185 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.231204 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.232283 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.232286 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.234131 (+ 1845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.234145 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.235228 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.235237 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.235542 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.236323 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:04.236345 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:04.238480 (+ 2135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:04.241539 (+ 3059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:04.241545 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:04.241554 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:04.241928 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:04.242244 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:04.242247 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:04.242249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:04.242409 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:04.242853 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:04.242858 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:04.243156 (+ 298us) client_negotiation.cc:769] Sending connection context
0809 04:45:04.243396 (+ 240us) client_negotiation.cc:240] Negotiation successful
0809 04:45:04.243748 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":297,"threads_started":1}
I20260809 04:45:04.246021 24705 heartbeater.cc:347] Connected to a master server at 127.12.234.62:33199
I20260809 04:45:04.246279 24705 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:04.246928 24705 heartbeater.cc:511] Master 127.12.234.62:33199 requested a full tablet report, sending...
I20260809 04:45:04.248391 24197 ts_manager.cc:194] Registered new tserver with Master: 65fdfbd0b0594136b96ad5e597f757b9 (127.12.234.3:34431)
I20260809 04:45:04.248924 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:45:04.249207 24197 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.3@KRBTEST.COM'} at 127.12.234.3:40063
I20260809 04:45:04.257872 24410 heartbeater.cc:503] Master 127.12.234.62:33199 was elected leader, sending a full tablet report...
I20260809 04:45:04.264674 24412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.253641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36754 (local address 127.12.234.62:33199)
0809 04:45:04.253773 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:04.253777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:04.254261 (+ 484us) server_negotiation.cc:407] Connection header received
0809 04:45:04.254369 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:04.254372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:04.254422 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:04.254490 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:04.255654 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.256303 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.257315 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.257657 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.257737 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.260016 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:04.260020 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:04.260023 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:04.260048 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:04.262580 (+ 2532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:04.263116 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:04.263120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:04.263121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:04.263245 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:04.263527 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:04.263529 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:04.263531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:04.263807 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:04.263981 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:04.264275 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 04:45:04.264445 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 04:45:04.268225 24197 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:45:04.274859 24197 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:45:04.296594 24718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.285508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:34431 (local address 127.0.0.1:43032)
0809 04:45:04.286686 (+ 1178us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:04.286729 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:04.286931 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:04.288264 (+ 1333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:04.288273 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:04.288309 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:04.288687 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.288706 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.291094 (+ 2388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.291099 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.293600 (+ 2501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.293614 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.295377 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.295383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.295593 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.295680 (+ 87us) client_negotiation.cc:769] Sending connection context
0809 04:45:04.295831 (+ 151us) client_negotiation.cc:240] Negotiation successful
0809 04:45:04.296024 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":881,"mutex_wait_us":28,"thread_start_us":714,"threads_started":1}
I20260809 04:45:04.297403 24720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.285825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43032 (local address 127.12.234.3:34431)
0809 04:45:04.287615 (+ 1790us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:04.287625 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:04.287661 (+ 36us) server_negotiation.cc:407] Connection header received
0809 04:45:04.287764 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:04.287772 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:04.287998 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:04.288885 (+ 887us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:04.289179 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.290932 (+ 1753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.293819 (+ 2887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.295153 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.295304 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.295556 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:04.296864 (+ 1308us) server_negotiation.cc:299] Negotiation successful
0809 04:45:04.297072 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1618,"thread_start_us":1457,"threads_started":1}
I20260809 04:45:04.298027 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.285804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:42731 (local address 127.0.0.1:34582)
0809 04:45:04.287881 (+ 2077us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:04.287897 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:04.287994 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:04.289089 (+ 1095us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:04.289093 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:04.289118 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:04.289572 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.289586 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.292173 (+ 2587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.292178 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.294846 (+ 2668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.294861 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.297194 (+ 2333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.297200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.297386 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.297425 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 04:45:04.297522 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 04:45:04.297660 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1946,"mutex_wait_us":138,"thread_start_us":6450,"threads_started":1}
I20260809 04:45:04.298707 24721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.286421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34582 (local address 127.12.234.2:42731)
0809 04:45:04.288501 (+ 2080us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:04.288511 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:04.288540 (+ 29us) server_negotiation.cc:407] Connection header received
0809 04:45:04.288644 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:04.288652 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:04.288855 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:04.289803 (+ 948us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:04.290012 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.291510 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.295017 (+ 3507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.296530 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.297116 (+ 586us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.297301 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:04.298243 (+ 942us) server_negotiation.cc:299] Negotiation successful
0809 04:45:04.298410 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1943,"thread_start_us":1844,"threads_started":1}
I20260809 04:45:04.301419 24719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.287036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:36017 (local address 127.0.0.1:35850)
0809 04:45:04.292693 (+ 5657us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:04.292711 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:04.292855 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:04.293323 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:04.293327 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:04.293355 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:04.293886 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.293897 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.295105 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.295109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.297492 (+ 2383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.298397 (+ 905us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.300547 (+ 2150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.300553 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.300788 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.300822 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 04:45:04.300922 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 04:45:04.301041 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5497,"mutex_wait_us":1,"thread_start_us":633,"threads_started":1}
I20260809 04:45:04.301470 24724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.287525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35850 (local address 127.12.234.1:36017)
0809 04:45:04.292054 (+ 4529us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:04.292063 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:04.292807 (+ 744us) server_negotiation.cc:407] Connection header received
0809 04:45:04.293010 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:04.293015 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:04.293165 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:04.293334 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:04.294050 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.294974 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.298577 (+ 3603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.300118 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.300276 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.300494 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:04.300984 (+ 490us) server_negotiation.cc:299] Negotiation successful
0809 04:45:04.301151 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4315,"thread_start_us":4190,"threads_started":1}
I20260809 04:45:04.302210 24627 tablet_service.cc:1467] Processing CreateTablet for tablet ad46b2ca449c4305b7e275af8e8acd50 (DEFAULT_TABLE table=test-table [id=b3a55757e3c94eb98ce8efd5b5622cee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:04.302109 24483 tablet_service.cc:1467] Processing CreateTablet for tablet ad46b2ca449c4305b7e275af8e8acd50 (DEFAULT_TABLE table=test-table [id=b3a55757e3c94eb98ce8efd5b5622cee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:04.303344 24627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad46b2ca449c4305b7e275af8e8acd50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:04.303428 24483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad46b2ca449c4305b7e275af8e8acd50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:04.304517 24336 tablet_service.cc:1467] Processing CreateTablet for tablet ad46b2ca449c4305b7e275af8e8acd50 (DEFAULT_TABLE table=test-table [id=b3a55757e3c94eb98ce8efd5b5622cee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:04.305714 24336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad46b2ca449c4305b7e275af8e8acd50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:04.314904 24725 tablet_bootstrap.cc:492] T ad46b2ca449c4305b7e275af8e8acd50 P 65fdfbd0b0594136b96ad5e597f757b9: Bootstrap starting.
I20260809 04:45:04.317173 24726 tablet_bootstrap.cc:492] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640: Bootstrap starting.
I20260809 04:45:04.318548 24725 tablet_bootstrap.cc:654] T ad46b2ca449c4305b7e275af8e8acd50 P 65fdfbd0b0594136b96ad5e597f757b9: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:04.319046 24727 tablet_bootstrap.cc:492] T ad46b2ca449c4305b7e275af8e8acd50 P d703b295100f4dd5848903cf5fb5d3bd: Bootstrap starting.
I20260809 04:45:04.320372 24725 log.cc:824] T ad46b2ca449c4305b7e275af8e8acd50 P 65fdfbd0b0594136b96ad5e597f757b9: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:04.321143 24726 tablet_bootstrap.cc:654] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:04.322255 24726 log.cc:824] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:04.322327 24727 tablet_bootstrap.cc:654] T ad46b2ca449c4305b7e275af8e8acd50 P d703b295100f4dd5848903cf5fb5d3bd: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:04.323390 24727 log.cc:824] T ad46b2ca449c4305b7e275af8e8acd50 P d703b295100f4dd5848903cf5fb5d3bd: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:04.323493 24725 tablet_bootstrap.cc:492] T ad46b2ca449c4305b7e275af8e8acd50 P 65fdfbd0b0594136b96ad5e597f757b9: No bootstrap required, opened a new log
I20260809 04:45:04.323771 24725 ts_tablet_manager.cc:1397] T ad46b2ca449c4305b7e275af8e8acd50 P 65fdfbd0b0594136b96ad5e597f757b9: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260809 04:45:04.325515 24726 tablet_bootstrap.cc:492] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640: No bootstrap required, opened a new log
I20260809 04:45:04.325747 24726 ts_tablet_manager.cc:1397] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 04:45:04.326156 24727 tablet_bootstrap.cc:492] T ad46b2ca449c4305b7e275af8e8acd50 P d703b295100f4dd5848903cf5fb5d3bd: No bootstrap required, opened a new log
I20260809 04:45:04.326391 24727 ts_tablet_manager.cc:1397] T ad46b2ca449c4305b7e275af8e8acd50 P d703b295100f4dd5848903cf5fb5d3bd: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 04:45:04.333218 24726 raft_consensus.cc:348] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65fdfbd0b0594136b96ad5e597f757b9" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 34431 } } peers { permanent_uuid: "fe443e30ed2b4349b99d5dcbe20e8640" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42731 } } peers { permanent_uuid: "d703b295100f4dd5848903cf5fb5d3bd" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36017 } }
I20260809 04:45:04.333551 24726 raft_consensus.cc:374] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:04.333667 24726 raft_consensus.cc:729] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe443e30ed2b4349b99d5dcbe20e8640, State: Initialized, Role: FOLLOWER
I20260809 04:45:04.334209 24726 consensus_queue.cc:260] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65fdfbd0b0594136b96ad5e597f757b9" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 34431 } } peers { permanent_uuid: "fe443e30ed2b4349b99d5dcbe20e8640" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42731 } } peers { permanent_uuid: "d703b295100f4dd5848903cf5fb5d3bd" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36017 } }
I20260809 04:45:04.334142 24725 raft_consensus.cc:348] T ad46b2ca449c4305b7e275af8e8acd50 P 65fdfbd0b0594136b96ad5e597f757b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65fdfbd0b0594136b96ad5e597f757b9" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 34431 } } peers { permanent_uuid: "fe443e30ed2b4349b99d5dcbe20e8640" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42731 } } peers { permanent_uuid: "d703b295100f4dd5848903cf5fb5d3bd" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36017 } }
I20260809 04:45:04.334509 24725 raft_consensus.cc:374] T ad46b2ca449c4305b7e275af8e8acd50 P 65fdfbd0b0594136b96ad5e597f757b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:04.334658 24725 raft_consensus.cc:729] T ad46b2ca449c4305b7e275af8e8acd50 P 65fdfbd0b0594136b96ad5e597f757b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65fdfbd0b0594136b96ad5e597f757b9, State: Initialized, Role: FOLLOWER
I20260809 04:45:04.335328 24725 consensus_queue.cc:260] T ad46b2ca449c4305b7e275af8e8acd50 P 65fdfbd0b0594136b96ad5e597f757b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65fdfbd0b0594136b96ad5e597f757b9" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 34431 } } peers { permanent_uuid: "fe443e30ed2b4349b99d5dcbe20e8640" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42731 } } peers { permanent_uuid: "d703b295100f4dd5848903cf5fb5d3bd" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36017 } }
I20260809 04:45:04.336735 24554 heartbeater.cc:503] Master 127.12.234.62:33199 was elected leader, sending a full tablet report...
I20260809 04:45:04.337275 24726 ts_tablet_manager.cc:1428] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260809 04:45:04.337958 24705 heartbeater.cc:503] Master 127.12.234.62:33199 was elected leader, sending a full tablet report...
I20260809 04:45:04.338281 24727 raft_consensus.cc:348] T ad46b2ca449c4305b7e275af8e8acd50 P d703b295100f4dd5848903cf5fb5d3bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65fdfbd0b0594136b96ad5e597f757b9" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 34431 } } peers { permanent_uuid: "fe443e30ed2b4349b99d5dcbe20e8640" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42731 } } peers { permanent_uuid: "d703b295100f4dd5848903cf5fb5d3bd" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36017 } }
I20260809 04:45:04.338661 24727 raft_consensus.cc:374] T ad46b2ca449c4305b7e275af8e8acd50 P d703b295100f4dd5848903cf5fb5d3bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:04.338744 24727 raft_consensus.cc:729] T ad46b2ca449c4305b7e275af8e8acd50 P d703b295100f4dd5848903cf5fb5d3bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d703b295100f4dd5848903cf5fb5d3bd, State: Initialized, Role: FOLLOWER
I20260809 04:45:04.338753 24725 ts_tablet_manager.cc:1428] T ad46b2ca449c4305b7e275af8e8acd50 P 65fdfbd0b0594136b96ad5e597f757b9: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260809 04:45:04.339352 24727 consensus_queue.cc:260] T ad46b2ca449c4305b7e275af8e8acd50 P d703b295100f4dd5848903cf5fb5d3bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65fdfbd0b0594136b96ad5e597f757b9" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 34431 } } peers { permanent_uuid: "fe443e30ed2b4349b99d5dcbe20e8640" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42731 } } peers { permanent_uuid: "d703b295100f4dd5848903cf5fb5d3bd" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36017 } }
I20260809 04:45:04.340937 24727 ts_tablet_manager.cc:1428] T ad46b2ca449c4305b7e275af8e8acd50 P d703b295100f4dd5848903cf5fb5d3bd: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260809 04:45:04.368618 24731 raft_consensus.cc:482] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:45:04.368846 24731 raft_consensus.cc:504] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65fdfbd0b0594136b96ad5e597f757b9" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 34431 } } peers { permanent_uuid: "fe443e30ed2b4349b99d5dcbe20e8640" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42731 } } peers { permanent_uuid: "d703b295100f4dd5848903cf5fb5d3bd" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36017 } }
I20260809 04:45:04.370518 24731 leader_election.cc:288] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 65fdfbd0b0594136b96ad5e597f757b9 (127.12.234.3:34431), d703b295100f4dd5848903cf5fb5d3bd (127.12.234.1:36017)
I20260809 04:45:04.379087 24724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.371600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:50473 (local address 127.12.234.1:36017)
0809 04:45:04.371875 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:04.371881 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:04.372164 (+ 283us) server_negotiation.cc:407] Connection header received
0809 04:45:04.372318 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:04.372321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:04.372379 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:04.372448 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:04.373083 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.374080 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.376628 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.377957 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.378118 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.378193 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:04.378723 (+ 530us) server_negotiation.cc:299] Negotiation successful
0809 04:45:04.378866 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260809 04:45:04.379083 24720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.370789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:54047 (local address 127.12.234.3:34431)
0809 04:45:04.370995 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:04.371003 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:04.371483 (+ 480us) server_negotiation.cc:407] Connection header received
0809 04:45:04.371797 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:04.371804 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:04.371901 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:04.371964 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:04.372731 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.373832 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.376636 (+ 2804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.377958 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.378113 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.378193 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:04.378723 (+ 530us) server_negotiation.cc:299] Negotiation successful
0809 04:45:04.378856 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:45:04.379334 24737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.371485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:36017 (local address 127.12.234.2:50473)
0809 04:45:04.372060 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:04.372105 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:04.372209 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:04.372493 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:04.372497 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:04.372524 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:04.372926 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.372934 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.374224 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.374230 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.376434 (+ 2204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.376442 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.378253 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.378259 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.378544 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.378603 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 04:45:04.378682 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 04:45:04.378859 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"mutex_wait_us":118,"thread_start_us":292,"threads_started":1}
I20260809 04:45:04.379592 24736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.370774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:34431 (local address 127.12.234.2:54047)
0809 04:45:04.371353 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:04.371389 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:04.371585 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:04.372205 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:04.372209 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:04.372228 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:04.372594 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.372606 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.373950 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.373956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.376418 (+ 2462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:04.376433 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.378258 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.378262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:04.378541 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.378595 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 04:45:04.378682 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 04:45:04.378850 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"mutex_wait_us":78,"thread_start_us":295,"threads_started":1}
I20260809 04:45:04.380136 24361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad46b2ca449c4305b7e275af8e8acd50" candidate_uuid: "fe443e30ed2b4349b99d5dcbe20e8640" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d703b295100f4dd5848903cf5fb5d3bd" is_pre_election: true
I20260809 04:45:04.380474 24653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad46b2ca449c4305b7e275af8e8acd50" candidate_uuid: "fe443e30ed2b4349b99d5dcbe20e8640" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65fdfbd0b0594136b96ad5e597f757b9" is_pre_election: true
I20260809 04:45:04.380640 24361 raft_consensus.cc:2454] T ad46b2ca449c4305b7e275af8e8acd50 P d703b295100f4dd5848903cf5fb5d3bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe443e30ed2b4349b99d5dcbe20e8640 in term 0.
I20260809 04:45:04.380955 24653 raft_consensus.cc:2454] T ad46b2ca449c4305b7e275af8e8acd50 P 65fdfbd0b0594136b96ad5e597f757b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe443e30ed2b4349b99d5dcbe20e8640 in term 0.
I20260809 04:45:04.381256 24434 leader_election.cc:302] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d703b295100f4dd5848903cf5fb5d3bd, fe443e30ed2b4349b99d5dcbe20e8640; no voters:
I20260809 04:45:04.381619 24731 raft_consensus.cc:2781] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:45:04.381755 24731 raft_consensus.cc:482] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:45:04.381863 24731 raft_consensus.cc:3037] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:04.384027 24731 raft_consensus.cc:504] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65fdfbd0b0594136b96ad5e597f757b9" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 34431 } } peers { permanent_uuid: "fe443e30ed2b4349b99d5dcbe20e8640" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42731 } } peers { permanent_uuid: "d703b295100f4dd5848903cf5fb5d3bd" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36017 } }
I20260809 04:45:04.384702 24731 leader_election.cc:288] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640 [CANDIDATE]: Term 1 election: Requested vote from peers 65fdfbd0b0594136b96ad5e597f757b9 (127.12.234.3:34431), d703b295100f4dd5848903cf5fb5d3bd (127.12.234.1:36017)
I20260809 04:45:04.385087 24653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad46b2ca449c4305b7e275af8e8acd50" candidate_uuid: "fe443e30ed2b4349b99d5dcbe20e8640" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65fdfbd0b0594136b96ad5e597f757b9"
I20260809 04:45:04.385159 24361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad46b2ca449c4305b7e275af8e8acd50" candidate_uuid: "fe443e30ed2b4349b99d5dcbe20e8640" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d703b295100f4dd5848903cf5fb5d3bd"
I20260809 04:45:04.385269 24653 raft_consensus.cc:3037] T ad46b2ca449c4305b7e275af8e8acd50 P 65fdfbd0b0594136b96ad5e597f757b9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:04.385378 24361 raft_consensus.cc:3037] T ad46b2ca449c4305b7e275af8e8acd50 P d703b295100f4dd5848903cf5fb5d3bd [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:04.387501 24653 raft_consensus.cc:2454] T ad46b2ca449c4305b7e275af8e8acd50 P 65fdfbd0b0594136b96ad5e597f757b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe443e30ed2b4349b99d5dcbe20e8640 in term 1.
I20260809 04:45:04.387524 24361 raft_consensus.cc:2454] T ad46b2ca449c4305b7e275af8e8acd50 P d703b295100f4dd5848903cf5fb5d3bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe443e30ed2b4349b99d5dcbe20e8640 in term 1.
I20260809 04:45:04.387866 24434 leader_election.cc:302] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65fdfbd0b0594136b96ad5e597f757b9, fe443e30ed2b4349b99d5dcbe20e8640; no voters:
I20260809 04:45:04.388134 24731 raft_consensus.cc:2781] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:04.388521 24731 raft_consensus.cc:686] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640 [term 1 LEADER]: Becoming Leader. State: Replica: fe443e30ed2b4349b99d5dcbe20e8640, State: Running, Role: LEADER
I20260809 04:45:04.388973 24731 consensus_queue.cc:237] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65fdfbd0b0594136b96ad5e597f757b9" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 34431 } } peers { permanent_uuid: "fe443e30ed2b4349b99d5dcbe20e8640" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42731 } } peers { permanent_uuid: "d703b295100f4dd5848903cf5fb5d3bd" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36017 } }
I20260809 04:45:04.393734 24197 catalog_manager.cc:5495] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640 reported cstate change: term changed from 0 to 1, leader changed from <none> to fe443e30ed2b4349b99d5dcbe20e8640 (127.12.234.2). New cstate: current_term: 1 leader_uuid: "fe443e30ed2b4349b99d5dcbe20e8640" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "65fdfbd0b0594136b96ad5e597f757b9" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 34431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe443e30ed2b4349b99d5dcbe20e8640" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 42731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d703b295100f4dd5848903cf5fb5d3bd" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36017 } health_report { overall_health: UNKNOWN } } }
I20260809 04:45:04.429312 24721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.424307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34590 (local address 127.12.234.2:42731)
0809 04:45:04.424486 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:04.424491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:04.424507 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:45:04.424668 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:04.424672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:04.424761 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:04.424888 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:45:04.425195 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.425957 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.427239 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.427656 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.427749 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.428237 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:45:04.428771 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:45:04.428913 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:04.428967 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 04:45:04.429030 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 04:45:04.437595 24361 raft_consensus.cc:1261] T ad46b2ca449c4305b7e275af8e8acd50 P d703b295100f4dd5848903cf5fb5d3bd [term 1 FOLLOWER]: Refusing update from remote peer fe443e30ed2b4349b99d5dcbe20e8640: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:04.437601 24653 raft_consensus.cc:1261] T ad46b2ca449c4305b7e275af8e8acd50 P 65fdfbd0b0594136b96ad5e597f757b9 [term 1 FOLLOWER]: Refusing update from remote peer fe443e30ed2b4349b99d5dcbe20e8640: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:04.438699 24738 consensus_queue.cc:1035] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d703b295100f4dd5848903cf5fb5d3bd" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:04.439072 24731 consensus_queue.cc:1035] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65fdfbd0b0594136b96ad5e597f757b9" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 34431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:04.450467 24741 mvcc.cc:205] Tried to move back new op lower bound from 7316482885365751808 to 7316482885181292544. Current Snapshot: MvccSnapshot[applied={T|T < 7316482885365751808}]
I20260809 04:45:04.451752 24742 mvcc.cc:205] Tried to move back new op lower bound from 7316482885365751808 to 7316482885181292544. Current Snapshot: MvccSnapshot[applied={T|T < 7316482885365751808}]
I20260809 04:45:04.465718 24196 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:04.466013 24196 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:04.470687 24196 catalog_manager.cc:5782] T 00000000000000000000000000000000 P afe2ada7846f419299fd145403c5dc66: Sending DeleteTablet for 3 replicas of tablet ad46b2ca449c4305b7e275af8e8acd50
I20260809 04:45:04.471972 24483 tablet_service.cc:1514] Processing DeleteTablet for tablet ad46b2ca449c4305b7e275af8e8acd50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:04 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:34582
I20260809 04:45:04.471971 24627 tablet_service.cc:1514] Processing DeleteTablet for tablet ad46b2ca449c4305b7e275af8e8acd50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:04 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:43032
I20260809 04:45:04.472389 24336 tablet_service.cc:1514] Processing DeleteTablet for tablet ad46b2ca449c4305b7e275af8e8acd50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:04 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:35850
I20260809 04:45:04.474915 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 24263
I20260809 04:45:04.483266 24753 tablet_replica.cc:330] T ad46b2ca449c4305b7e275af8e8acd50 P d703b295100f4dd5848903cf5fb5d3bd: stopping tablet replica
I20260809 04:45:04.483596 24750 tablet_replica.cc:330] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640: stopping tablet replica
I20260809 04:45:04.484001 24753 raft_consensus.cc:2229] T ad46b2ca449c4305b7e275af8e8acd50 P d703b295100f4dd5848903cf5fb5d3bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:04.484120 24750 raft_consensus.cc:2229] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:45:04.484649 24753 raft_consensus.cc:2258] T ad46b2ca449c4305b7e275af8e8acd50 P d703b295100f4dd5848903cf5fb5d3bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:04.484977 24750 raft_consensus.cc:2258] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:04.486606 24753 ts_tablet_manager.cc:1905] T ad46b2ca449c4305b7e275af8e8acd50 P d703b295100f4dd5848903cf5fb5d3bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:04.487689 24750 ts_tablet_manager.cc:1905] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:04.494311 24753 ts_tablet_manager.cc:1918] T ad46b2ca449c4305b7e275af8e8acd50 P d703b295100f4dd5848903cf5fb5d3bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:45:04.494612 24753 log.cc:1196] T ad46b2ca449c4305b7e275af8e8acd50 P d703b295100f4dd5848903cf5fb5d3bd: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-0/wal/wals/ad46b2ca449c4305b7e275af8e8acd50
I20260809 04:45:04.495249 24753 ts_tablet_manager.cc:1939] T ad46b2ca449c4305b7e275af8e8acd50 P d703b295100f4dd5848903cf5fb5d3bd: Deleting consensus metadata
I20260809 04:45:04.497169 24180 catalog_manager.cc:4841] TS d703b295100f4dd5848903cf5fb5d3bd (127.12.234.1:36017): tablet ad46b2ca449c4305b7e275af8e8acd50 (table test-table [id=b3a55757e3c94eb98ce8efd5b5622cee]) successfully deleted
I20260809 04:45:04.497272 24750 ts_tablet_manager.cc:1918] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:45:04.497493 24750 log.cc:1196] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-1/wal/wals/ad46b2ca449c4305b7e275af8e8acd50
I20260809 04:45:04.498282 24750 ts_tablet_manager.cc:1939] T ad46b2ca449c4305b7e275af8e8acd50 P fe443e30ed2b4349b99d5dcbe20e8640: Deleting consensus metadata
I20260809 04:45:04.500631 24751 tablet_replica.cc:330] T ad46b2ca449c4305b7e275af8e8acd50 P 65fdfbd0b0594136b96ad5e597f757b9: stopping tablet replica
I20260809 04:45:04.501019 24751 raft_consensus.cc:2229] T ad46b2ca449c4305b7e275af8e8acd50 P 65fdfbd0b0594136b96ad5e597f757b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:04.501502 24751 raft_consensus.cc:2258] T ad46b2ca449c4305b7e275af8e8acd50 P 65fdfbd0b0594136b96ad5e597f757b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:04.502712 24178 catalog_manager.cc:4841] TS fe443e30ed2b4349b99d5dcbe20e8640 (127.12.234.2:42731): tablet ad46b2ca449c4305b7e275af8e8acd50 (table test-table [id=b3a55757e3c94eb98ce8efd5b5622cee]) successfully deleted
I20260809 04:45:04.503348 24751 ts_tablet_manager.cc:1905] T ad46b2ca449c4305b7e275af8e8acd50 P 65fdfbd0b0594136b96ad5e597f757b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:04.507833 24751 ts_tablet_manager.cc:1918] T ad46b2ca449c4305b7e275af8e8acd50 P 65fdfbd0b0594136b96ad5e597f757b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:45:04.507988 24751 log.cc:1196] T ad46b2ca449c4305b7e275af8e8acd50 P 65fdfbd0b0594136b96ad5e597f757b9: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786250603326224-13224-0/minicluster-data/ts-2/wal/wals/ad46b2ca449c4305b7e275af8e8acd50
I20260809 04:45:04.508612 24751 ts_tablet_manager.cc:1939] T ad46b2ca449c4305b7e275af8e8acd50 P 65fdfbd0b0594136b96ad5e597f757b9: Deleting consensus metadata
I20260809 04:45:04.510365 24180 catalog_manager.cc:4841] TS 65fdfbd0b0594136b96ad5e597f757b9 (127.12.234.3:34431): tablet ad46b2ca449c4305b7e275af8e8acd50 (table test-table [id=b3a55757e3c94eb98ce8efd5b5622cee]) successfully deleted
Aug 09 04:45:04 dist-test-slave-1jjb krb5kdc[24148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.1@KRBTEST.COM
I20260809 04:45:04.519114 24724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.476188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35854 (local address 127.12.234.1:36017)
0809 04:45:04.476448 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:04.476457 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:04.490778 (+ 14321us) server_negotiation.cc:407] Connection header received
0809 04:45:04.491256 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:04.491263 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:04.491350 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:04.491438 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:04.493270 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.494361 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.508103 (+ 13742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.508896 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.509066 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.513751 (+ 4685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:04.513761 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:04.513774 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:04.513803 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:04.516497 (+ 2694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:04.517238 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:04.517243 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:04.517245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:04.517380 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:04.517810 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:04.517814 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:04.517816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:04.518146 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:04.518384 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:04.518684 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 04:45:04.518822 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 04:45:04.519798 24404 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:45:04.611591 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 24263
I20260809 04:45:04.635613 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 24416
Aug 09 04:45:04 dist-test-slave-1jjb krb5kdc[24148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.2@KRBTEST.COM
I20260809 04:45:04.651021 24721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.636411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34604 (local address 127.12.234.2:42731)
0809 04:45:04.636567 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:04.636573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:04.636619 (+ 46us) server_negotiation.cc:407] Connection header received
0809 04:45:04.636817 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:04.636823 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:04.636891 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:04.636993 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:04.638238 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.639029 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.640181 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.640647 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.640780 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.644889 (+ 4109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:04.644903 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:04.644924 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:04.644965 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:04.647932 (+ 2967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:04.648739 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:04.648754 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:04.648757 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:04.648917 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:04.649371 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:04.649375 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:04.649377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:04.649752 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:04.650146 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:04.650441 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 04:45:04.650674 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:45:04.651791 24549 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:45:04.760818 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 24416
I20260809 04:45:04.782477 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 24559
Aug 09 04:45:04 dist-test-slave-1jjb krb5kdc[24148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.3@KRBTEST.COM
I20260809 04:45:04.797078 24720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:04.783251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43042 (local address 127.12.234.3:34431)
0809 04:45:04.783458 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:04.783465 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:04.783745 (+ 280us) server_negotiation.cc:407] Connection header received
0809 04:45:04.784030 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:04.784036 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:04.784155 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:04.784281 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:04.785483 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.786198 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.787390 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:04.787734 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:04.787814 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:04.791998 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:04.792010 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:04.792026 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:04.792056 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:04.794592 (+ 2536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:04.795202 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:04.795208 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:04.795210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:04.795349 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:04.795677 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:04.795680 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:04.795682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:04.796008 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:04.796277 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:04.796586 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 04:45:04.796750 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 04:45:04.797752 24700 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:45:04.883176 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 24559
I20260809 04:45:04.902938 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 24161
I20260809 04:45:04.904023 24231 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:45:05.006155 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 24161
2026-08-09T04:45:05Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4161 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:45:05 dist-test-slave-1jjb krb5kdc[24761](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:45:05 dist-test-slave-1jjb krb5kdc[24761](info): set up 2 sockets
Aug 09 04:45:05 dist-test-slave-1jjb krb5kdc[24761](info): commencing operation
krb5kdc: starting...
W20260809 04:45:06.094092 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.002s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:45:06 dist-test-slave-1jjb krb5kdc[24761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:45:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:45:06Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.12.234.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.12.234.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.12.234.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/rangeradmin_127.12.234.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.12.234.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/rangeradmin_127.12.234.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.12.234.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.12.234.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.12.234.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/rangerlookup_127.12.234.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.12.234.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/rangerlookup_127.12.234.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.234.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.12.234.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.234.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/HTTP_127.12.234.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.234.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/HTTP_127.12.234.20@KRBTEST.COM.keytab.
I20260809 04:45:06.431629 13224 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskQ3A_6j/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/postgres -l logfile start
2026-08-09 04:45:08.787 UTC [24795] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 04:45:08.787 UTC [24795] LOG: listening on IPv4 address "127.12.234.20", port 55737
2026-08-09 04:45:08.791 UTC [24795] LOG: listening on Unix socket "/tmp/.s.PGSQL.55737"
2026-08-09 04:45:08.814 UTC [24798] LOG: database system was shut down at 2026-08-09 04:45:07 UTC
2026-08-09 04:45:08.817 UTC [24795] LOG: database system is ready to accept connections
I20260809 04:45:09.794000 13224 mini_postgres.cc:96] Postgres bound to 55737
2026-08-09 04:45:09.819 UTC [24807] FATAL: database "slave" does not exist
127.12.234.20:55737 - accepting connections
I20260809 04:45:09.820199 13224 mini_ranger.cc:161] Starting Ranger...
I20260809 04:45:09.851315 13224 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 04:45:10.288852 13224 mini_ranger.cc:87] Created ranger Postgres database
I20260809 04:45:10.289048 13224 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ranger-admin
2026-08-09 04:45:10,624 [I] DB FLAVOR :POSTGRES
2026-08-09 04:45:10,624 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 04:45:10,625 [I] Checking connection..
2026-08-09 04:45:10,625 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:55737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 04:45:10,863 [I] Checking connection passed.
2026-08-09 04:45:10,863 [I] --------- Verifying version history table ---------
2026-08-09 04:45:10,864 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:55737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 04:45:11,107 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 04:45:11,108 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:55737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 04:45:11,356 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 04:45:11,356 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 04:45:11,356 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:55737/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 04:45:11.594 UTC [24875] WARNING: there is no transaction in progress
2026-08-09 04:45:11,602 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 04:45:11,602 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:55737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 04:45:11,831 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 04:45:11,832 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 04:45:11,832 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:55737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 04:45:12,049 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:55737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 04:45:12,292 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:55737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1jjb.c.gcp-upstream.internal','N') ;"
2026-08-09 04:45:12,528 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:55737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 04:45:12,771 [I] Table x_portal_user does not exist in database ranger
2026-08-09 04:45:12,772 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:55737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 04:45:13,011 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 04:45:13,011 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:55737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 04:45:13,248 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:55737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 04:45:13,486 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 04:45:13,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:55737/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 04:45:14.368 UTC [24996] WARNING: there is no transaction in progress
2026-08-09 04:45:14.383 UTC [24996] WARNING: there is no transaction in progress
2026-08-09 04:45:14.397 UTC [24996] WARNING: there is no transaction in progress
2026-08-09 04:45:14.412 UTC [24996] WARNING: there is no transaction in progress
2026-08-09 04:45:14.427 UTC [24996] WARNING: there is no transaction in progress
2026-08-09 04:45:14.443 UTC [24996] WARNING: there is no transaction in progress
2026-08-09 04:45:14.504 UTC [24996] WARNING: there is no transaction in progress
2026-08-09 04:45:14.509 UTC [24996] WARNING: there is no transaction in progress
2026-08-09 04:45:14.522 UTC [24996] WARNING: there is no transaction in progress
2026-08-09 04:45:14.531 UTC [24996] WARNING: there is no transaction in progress
2026-08-09 04:45:14.540 UTC [24996] WARNING: there is no transaction in progress
2026-08-09 04:45:14.552 UTC [24996] WARNING: there is no transaction in progress
2026-08-09 04:45:14.561 UTC [24996] WARNING: there is no transaction in progress
2026-08-09 04:45:14.567 UTC [24996] WARNING: there is no transaction in progress
2026-08-09 04:45:14.572 UTC [24996] WARNING: there is no transaction in progress
2026-08-09 04:45:14.578 UTC [24996] WARNING: there is no transaction in progress
2026-08-09 04:45:15,150 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 04:45:15,151 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:55737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 04:45:15,393 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 04:45:15,393 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:55737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 04:45:15,646 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 04:45:15,647 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:55737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 04:45:15,884 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 04:45:15,884 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:55737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 04:45:16,131 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:55737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 04:45:16,363 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:55737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1jjb.c.gcp-upstream.internal';"
2026-08-09 04:45:16,584 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 04:45:16,584 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:55737/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 04:45:16,808 [I] DB_PATCHES have already been applied
I20260809 04:45:16.814175 13224 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ranger-admin:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskQ3A_6j/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 04:45:16.814260 13224 mini_ranger.cc:193] Using host: 127.12.234.20
I20260809 04:45:16.835837 13224 mini_ranger.cc:227] Ranger admin URL: http://127.12.234.20:51337
Aug 09, 2026 4:45:17 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:45:17 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:45:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 4:45:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 4:45:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 4:45:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 4:45:17 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 04:45:17.973382 13224 mini_ranger.cc:160] Time spent starting Ranger: real 8.153s user 0.000s sys 0.060s
Aug 09, 2026 4:45:37 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 4:45:37 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 09, 2026 4:45:37 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 4:45:37 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 4:45:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 4:45:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 4:45:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 4:45:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 4:45:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 4:45:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 4:45:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 4:45:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 4:45:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 4:45:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 4:45:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 4:45:37 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 4:45:37 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 4:45:38 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 04:45:38.518946 13224 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.12.234.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.12.234.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.12.234.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/rangerkms_127.12.234.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.12.234.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/rangerkms_127.12.234.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.234.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.12.234.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.234.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/HTTP_127.12.234.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.234.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/HTTP_127.12.234.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 04:45:38 dist-test-slave-1jjb krb5kdc[24761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250738, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 04:45:38.719388 13224 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 04:45:38.742017 13224 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 04:45:39.568951 13224 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 04:45:39.569159 13224 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ranger-kms
I20260809 04:45:39.569293 13224 mini_ranger_kms.cc:227] Using postgres at 127.12.234.20:55737
2026-08-09 04:45:40,370 [I] DB FLAVOR :POSTGRES
2026-08-09 04:45:40,370 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 04:45:40,370 [I] Checking connection
2026-08-09 04:45:40,370 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:55737/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 04:45:40,610 [I] connection success
2026-08-09 04:45:40,611 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 04:45:40,611 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 04:45:40,611 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:55737/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 04:45:40,847 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 04:45:40,848 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 04:45:40,848 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 04:45:40,848 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:55737/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 04:45:41,120 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 04:45:41.591701 13224 mini_ranger_kms.cc:310] Created kms service
I20260809 04:45:41.817050 13224 mini_ranger_kms.cc:326] Created kudu user
I20260809 04:45:41.860906 13224 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 04:45:41 dist-test-slave-1jjb krb5kdc[24761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250738, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.12.234.20@KRBTEST.COM
I20260809 04:45:42.268813 13224 mini_ranger_kms.cc:384] Added ranger policy
I20260809 04:45:42.270264 13224 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ranger-kms:/tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 04:45:42.270337 13224 mini_ranger_kms.cc:240] Using host: 127.12.234.20
I20260809 04:45:42.297593 13224 mini_ranger_kms.cc:276] Ranger KMS PID: 25193
I20260809 04:45:42.297768 13224 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.12.234.20:54093
Aug 09, 2026 4:45:43 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:45:43 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:45:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 4:45:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 4:45:43 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 4:45:43 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260809 04:45:44.652119 13224 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.933s user 0.171s sys 0.418s
I20260809 04:45:44.652470 13224 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 04:45:44.652715 13224 mini_ranger_kms.cc:401] 127.12.234.20:54093/kms/v1/keys
Aug 09 04:45:44 dist-test-slave-1jjb krb5kdc[24761](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786250744, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.234.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 04:45:45 dist-test-slave-1jjb krb5kdc[24761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250744, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.234.20@KRBTEST.COM for HTTP/127.12.234.20@KRBTEST.COM
Aug 09 04:45:45 dist-test-slave-1jjb krb5kdc[24761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250744, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.234.20@KRBTEST.COM for HTTP/127.12.234.20@KRBTEST.COM
Aug 09 04:45:48 dist-test-slave-1jjb krb5kdc[24761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:45:48.145813 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:44859
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:33283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:44859
--ranger_config_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.234.20:54093/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:48.798292 25252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:48.798674 25252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:48.798867 25252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:48.806478 25252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:45:48.806757 25252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:48.806941 25252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:48.807057 25252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:45:48.807163 25252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:45:48.820570 25252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:33283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/master-0/wal
--ranger_kms_url=127.12.234.20:54093/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:44859
--ranger_config_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:44859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:45:48.822202 25252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:48.823717 25252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:48.842026 25257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:48.842820 25258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:48.843137 25260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:48.843566 25252 server_base.cc:1034] running on GCE node
I20260809 04:45:48.844611 25252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:48.846139 25252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:48.847429 25252 hybrid_clock.cc:648] HybridClock initialized: now 1786250748847304 us; error 32 us; skew 500 ppm
I20260809 04:45:48.847904 25252 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:45:48 dist-test-slave-1jjb krb5kdc[24761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250748, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:48.853036 25252 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:45:48.855345 25252 webserver.cc:460] Webserver started at http://127.12.234.62:43741/ using document root <none> and password file <none>
I20260809 04:45:48.856266 25252 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:48.856511 25252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:48.856928 25252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:45:48 dist-test-slave-1jjb krb5kdc[24761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250748, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for HTTP/127.12.234.20@KRBTEST.COM
I20260809 04:45:48.925915 25252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "42d0b6e8cb584a3b841961e050f24643"
format_stamp: "Formatted at 2026-08-09 04:45:48 on dist-test-slave-1jjb"
server_key: "e4613e35445779ede92736aeaf3ddf58"
server_key_iv: "e80b240d21e38df87a87251ca2e1c300"
server_key_version: "kuduclusterkey@0"
I20260809 04:45:48.926885 25252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "42d0b6e8cb584a3b841961e050f24643"
format_stamp: "Formatted at 2026-08-09 04:45:48 on dist-test-slave-1jjb"
server_key: "e4613e35445779ede92736aeaf3ddf58"
server_key_iv: "e80b240d21e38df87a87251ca2e1c300"
server_key_version: "kuduclusterkey@0"
I20260809 04:45:48.934610 25252 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260809 04:45:48.957810 25268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:48.959955 25252 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260809 04:45:48.960309 25252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "42d0b6e8cb584a3b841961e050f24643"
format_stamp: "Formatted at 2026-08-09 04:45:48 on dist-test-slave-1jjb"
server_key: "e4613e35445779ede92736aeaf3ddf58"
server_key_iv: "e80b240d21e38df87a87251ca2e1c300"
server_key_version: "kuduclusterkey@0"
I20260809 04:45:48.960736 25252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:48.998672 25252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:49.004408 25252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:49.004895 25252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:49.050863 25252 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:44859
I20260809 04:45:49.051064 25324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:44859 every 8 connection(s)
I20260809 04:45:49.052731 25252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:45:49.057466 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 25252
I20260809 04:45:49.057679 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:45:49.058419 13224 external_mini_cluster.cc:1428] Setting key ce4b141f6e7d53c7c30d1c848517f572
I20260809 04:45:49.058851 25325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:49.074189 25325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42d0b6e8cb584a3b841961e050f24643: Bootstrap starting.
Aug 09 04:45:49 dist-test-slave-1jjb krb5kdc[24761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:45:49.077612 25325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42d0b6e8cb584a3b841961e050f24643: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:49.079350 25325 log.cc:824] T 00000000000000000000000000000000 P 42d0b6e8cb584a3b841961e050f24643: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:49.083313 25325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42d0b6e8cb584a3b841961e050f24643: No bootstrap required, opened a new log
I20260809 04:45:49.087729 25328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:49.060939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35514 (local address 127.12.234.62:44859)
0809 04:45:49.062428 (+ 1489us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:49.062440 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:49.062488 (+ 48us) server_negotiation.cc:407] Connection header received
0809 04:45:49.066423 (+ 3935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:49.066458 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:49.066909 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:49.067613 (+ 704us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:49.069063 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:49.070608 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:49.071861 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:49.072872 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:49.073068 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:49.077651 (+ 4583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:49.077686 (+ 35us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:49.077709 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:49.077794 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:49.081996 (+ 4202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:49.082970 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:49.082982 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:49.082991 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:49.083214 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:49.083816 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:49.083825 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:49.083827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:49.084845 (+ 1018us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:49.085151 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:49.085696 (+ 545us) server_negotiation.cc:299] Negotiation successful
0809 04:45:49.086105 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1177,"thread_start_us":874,"threads_started":1}
I20260809 04:45:49.096675 25325 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42d0b6e8cb584a3b841961e050f24643 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d0b6e8cb584a3b841961e050f24643" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44859 } }
I20260809 04:45:49.097184 25325 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42d0b6e8cb584a3b841961e050f24643 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:49.097354 25325 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42d0b6e8cb584a3b841961e050f24643 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42d0b6e8cb584a3b841961e050f24643, State: Initialized, Role: FOLLOWER
I20260809 04:45:49.098125 25325 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42d0b6e8cb584a3b841961e050f24643 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d0b6e8cb584a3b841961e050f24643" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44859 } }
I20260809 04:45:49.098483 25325 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42d0b6e8cb584a3b841961e050f24643 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:45:49.098762 25325 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42d0b6e8cb584a3b841961e050f24643 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:45:49.099021 25325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42d0b6e8cb584a3b841961e050f24643 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:49.102516 25325 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42d0b6e8cb584a3b841961e050f24643 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d0b6e8cb584a3b841961e050f24643" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44859 } }
I20260809 04:45:49.103193 25325 leader_election.cc:302] T 00000000000000000000000000000000 P 42d0b6e8cb584a3b841961e050f24643 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42d0b6e8cb584a3b841961e050f24643; no voters:
I20260809 04:45:49.104399 25325 leader_election.cc:288] T 00000000000000000000000000000000 P 42d0b6e8cb584a3b841961e050f24643 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:45:49.104516 25330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42d0b6e8cb584a3b841961e050f24643 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:49.106611 25330 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42d0b6e8cb584a3b841961e050f24643 [term 1 LEADER]: Becoming Leader. State: Replica: 42d0b6e8cb584a3b841961e050f24643, State: Running, Role: LEADER
I20260809 04:45:49.107442 25330 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42d0b6e8cb584a3b841961e050f24643 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d0b6e8cb584a3b841961e050f24643" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44859 } }
I20260809 04:45:49.108120 25325 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42d0b6e8cb584a3b841961e050f24643 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:45:49.111881 25331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42d0b6e8cb584a3b841961e050f24643 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42d0b6e8cb584a3b841961e050f24643" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d0b6e8cb584a3b841961e050f24643" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44859 } } }
I20260809 04:45:49.111780 25332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42d0b6e8cb584a3b841961e050f24643 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42d0b6e8cb584a3b841961e050f24643. Latest consensus state: current_term: 1 leader_uuid: "42d0b6e8cb584a3b841961e050f24643" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42d0b6e8cb584a3b841961e050f24643" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44859 } } }
I20260809 04:45:49.113118 25331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42d0b6e8cb584a3b841961e050f24643 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:49.113118 25332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42d0b6e8cb584a3b841961e050f24643 [sys.catalog]: This master's current role is: LEADER
I20260809 04:45:49.117494 25336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:45:49.127040 25336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:45:49.134914 25325 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 04:45:49.157548 25336 catalog_manager.cc:1347] Generated new cluster ID: 1c706a015b514b9ab9674c8e0ab60f85
I20260809 04:45:49.157732 25336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:45:49.175305 25336 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:45:49.178090 25336 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:45:49.196981 25336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42d0b6e8cb584a3b841961e050f24643: Generated new TSK 0
I20260809 04:45:49.198968 25336 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
I20260809 04:45:50.155637 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.234.20:54093/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:0
--local_ip_for_outbound_sockets=127.12.234.1
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44859
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:33283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:45:50 dist-test-slave-1jjb krb5kdc[24761](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786250750, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 04:45:50.910866 25376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:50.911491 25376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:50.911839 25376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:50.926090 25376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:50.926426 25376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:50.926764 25376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:45:50.944041 25376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:33283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.12.234.20:54093/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:45:50.945611 25376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:50.947721 25376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:50.985994 25387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:50.989696 25386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:50.993794 25376 server_base.cc:1034] running on GCE node
W20260809 04:45:50.998168 25389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:50.999250 25376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:51.006577 25376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:51.008608 25376 hybrid_clock.cc:648] HybridClock initialized: now 1786250751007826 us; error 126 us; skew 500 ppm
I20260809 04:45:51.009373 25376 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:45:51 dist-test-slave-1jjb krb5kdc[24761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250751, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:51.016286 25376 init.cc:377] Logged in from keytab as kudu/127.12.234.1@KRBTEST.COM (short username kudu)
Aug 09 04:45:51 dist-test-slave-1jjb krb5kdc[24761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250750, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for HTTP/127.12.234.20@KRBTEST.COM
I20260809 04:45:51.032778 25376 webserver.cc:460] Webserver started at http://127.12.234.1:45797/ using document root <none> and password file <none>
I20260809 04:45:51.034343 25376 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:51.034719 25376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:51.035589 25376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:45:51 dist-test-slave-1jjb krb5kdc[24761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250751, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for HTTP/127.12.234.20@KRBTEST.COM
I20260809 04:45:51.076092 25376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-0/data/instance:
uuid: "5df9343ac86748e588ab079c662bb767"
format_stamp: "Formatted at 2026-08-09 04:45:51 on dist-test-slave-1jjb"
server_key: "3ec81b95011ce1ed09c5517f610d2e48"
server_key_iv: "14f0874a9e184d6255480e4dfcb5712e"
server_key_version: "kuduclusterkey@0"
I20260809 04:45:51.077634 25376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance:
uuid: "5df9343ac86748e588ab079c662bb767"
format_stamp: "Formatted at 2026-08-09 04:45:51 on dist-test-slave-1jjb"
server_key: "3ec81b95011ce1ed09c5517f610d2e48"
server_key_iv: "14f0874a9e184d6255480e4dfcb5712e"
server_key_version: "kuduclusterkey@0"
I20260809 04:45:51.095975 25376 fs_manager.cc:691] Time spent creating directory manager: real 0.018s user 0.004s sys 0.013s
I20260809 04:45:51.131165 25398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:51.134717 25376 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.004s sys 0.000s
I20260809 04:45:51.135075 25376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "5df9343ac86748e588ab079c662bb767"
format_stamp: "Formatted at 2026-08-09 04:45:51 on dist-test-slave-1jjb"
server_key: "3ec81b95011ce1ed09c5517f610d2e48"
server_key_iv: "14f0874a9e184d6255480e4dfcb5712e"
server_key_version: "kuduclusterkey@0"
I20260809 04:45:51.135824 25376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:51.174012 25376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:51.183553 25376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:51.184412 25376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:51.187084 25376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:51.190328 25376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:51.190550 25376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:51.190861 25376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:51.191044 25376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:51.354900 25376 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:36689
I20260809 04:45:51.355005 25521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:36689 every 8 connection(s)
I20260809 04:45:51.356949 25376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:45:51.364410 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 25376
I20260809 04:45:51.364711 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance
I20260809 04:45:51.366004 13224 external_mini_cluster.cc:1428] Setting key 14e231bf2b36cbc723ef7b554b270462
Aug 09 04:45:51 dist-test-slave-1jjb krb5kdc[24761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250751, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:45:51.407784 25527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:51.363002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44859 (local address 127.12.234.1:39215)
0809 04:45:51.368739 (+ 5737us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:51.368787 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:51.371997 (+ 3210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:51.373495 (+ 1498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:51.373514 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:51.374894 (+ 1380us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:51.376293 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:51.376325 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:51.378491 (+ 2166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:51.378499 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:51.381719 (+ 3220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:51.381738 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:51.383648 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:51.383660 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:51.384558 (+ 898us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:51.386023 (+ 1465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:51.386095 (+ 72us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:51.391956 (+ 5861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:51.399904 (+ 7948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:51.399917 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:51.399946 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:51.401023 (+ 1077us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:51.401842 (+ 819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:51.401851 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:51.401858 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:51.402393 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:51.403637 (+ 1244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:51.403651 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:51.404321 (+ 670us) client_negotiation.cc:769] Sending connection context
0809 04:45:51.404791 (+ 470us) client_negotiation.cc:240] Negotiation successful
0809 04:45:51.405668 (+ 877us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4203,"thread_start_us":4427,"threads_started":1}
I20260809 04:45:51.410036 25526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:51.361826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:39215 (local address 127.12.234.62:44859)
0809 04:45:51.368636 (+ 6810us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:51.368643 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:51.369089 (+ 446us) server_negotiation.cc:407] Connection header received
0809 04:45:51.372551 (+ 3462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:51.372556 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:51.372646 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:51.372964 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:51.376529 (+ 3565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:51.378232 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:51.381898 (+ 3666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:51.382971 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:51.383094 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:51.392418 (+ 9324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:51.392429 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:51.392436 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:51.392521 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:51.399717 (+ 7196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:51.401255 (+ 1538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:51.401274 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:51.401279 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:51.401669 (+ 390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:51.402626 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:51.402636 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:51.402643 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:51.403297 (+ 654us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:51.403521 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:51.408795 (+ 5274us) server_negotiation.cc:299] Negotiation successful
0809 04:45:51.409176 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6623,"thread_start_us":10285,"threads_started":1}
I20260809 04:45:51.411804 25522 heartbeater.cc:347] Connected to a master server at 127.12.234.62:44859
I20260809 04:45:51.412374 25522 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:51.413435 25522 heartbeater.cc:511] Master 127.12.234.62:44859 requested a full tablet report, sending...
I20260809 04:45:51.416656 25288 ts_manager.cc:194] Registered new tserver with Master: 5df9343ac86748e588ab079c662bb767 (127.12.234.1:36689)
I20260809 04:45:51.419740 25288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.1@KRBTEST.COM'} at 127.12.234.1:39215
WARNING: no policy specified for kudu/127.12.234.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
W20260809 04:45:51.488188 13224 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.12.234.2: real 0.113s user 0.002s sys 0.033s
I20260809 04:45:51.489236 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.234.20:54093/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:0
--local_ip_for_outbound_sockets=127.12.234.2
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44859
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:33283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:45:51 dist-test-slave-1jjb krb5kdc[24761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250750, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for HTTP/127.12.234.20@KRBTEST.COM
W20260809 04:45:52.289695 25531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:52.290078 25531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:52.290228 25531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:52.301148 25531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:52.301344 25531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:52.301522 25531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:45:52.315635 25531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:33283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.12.234.20:54093/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:45:52.317221 25531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:52.318632 25531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:52.340333 25545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:52.340482 25546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:52.340605 25548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:52.342643 25531 server_base.cc:1034] running on GCE node
I20260809 04:45:52.343297 25531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:52.344512 25531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:52.345809 25531 hybrid_clock.cc:648] HybridClock initialized: now 1786250752345687 us; error 54 us; skew 500 ppm
I20260809 04:45:52.346203 25531 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:45:52 dist-test-slave-1jjb krb5kdc[24761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250752, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:52.351212 25531 init.cc:377] Logged in from keytab as kudu/127.12.234.2@KRBTEST.COM (short username kudu)
I20260809 04:45:52.353848 25531 webserver.cc:460] Webserver started at http://127.12.234.2:45925/ using document root <none> and password file <none>
I20260809 04:45:52.354746 25531 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:52.354990 25531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:52.355439 25531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:45:52 dist-test-slave-1jjb krb5kdc[24761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250752, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for HTTP/127.12.234.20@KRBTEST.COM
I20260809 04:45:52.374624 25531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-1/data/instance:
uuid: "2d54543f1f3443bbb10c25b74d9de956"
format_stamp: "Formatted at 2026-08-09 04:45:52 on dist-test-slave-1jjb"
server_key: "2eafa78a1bf1c6173ae60f33a356d499"
server_key_iv: "365007220397156d099c557847787884"
server_key_version: "kuduclusterkey@0"
I20260809 04:45:52.375918 25531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance:
uuid: "2d54543f1f3443bbb10c25b74d9de956"
format_stamp: "Formatted at 2026-08-09 04:45:52 on dist-test-slave-1jjb"
server_key: "2eafa78a1bf1c6173ae60f33a356d499"
server_key_iv: "365007220397156d099c557847787884"
server_key_version: "kuduclusterkey@0"
I20260809 04:45:52.383528 25531 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260809 04:45:52.401423 25555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:52.403290 25531 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260809 04:45:52.403568 25531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "2d54543f1f3443bbb10c25b74d9de956"
format_stamp: "Formatted at 2026-08-09 04:45:52 on dist-test-slave-1jjb"
server_key: "2eafa78a1bf1c6173ae60f33a356d499"
server_key_iv: "365007220397156d099c557847787884"
server_key_version: "kuduclusterkey@0"
I20260809 04:45:52.403981 25531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:52.423794 25522 heartbeater.cc:503] Master 127.12.234.62:44859 was elected leader, sending a full tablet report...
I20260809 04:45:52.433955 25531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:52.440050 25531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:52.440752 25531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:52.442860 25531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:52.446564 25531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:52.446748 25531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:52.447014 25531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:52.447186 25531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:52.545334 25531 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:46615
I20260809 04:45:52.545431 25682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:46615 every 8 connection(s)
I20260809 04:45:52.547096 25531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:45:52.549223 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 25531
I20260809 04:45:52.549427 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance
I20260809 04:45:52.550058 13224 external_mini_cluster.cc:1428] Setting key 04858da031dbec3d10cc2519897cfeb3
Aug 09 04:45:52 dist-test-slave-1jjb krb5kdc[24761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250752, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:45:52.579825 25685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:52.551871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:52935 (local address 127.12.234.62:44859)
0809 04:45:52.552960 (+ 1089us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:52.552964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:52.554656 (+ 1692us) server_negotiation.cc:407] Connection header received
0809 04:45:52.557793 (+ 3137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:52.557798 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:52.557897 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:52.558059 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:52.560279 (+ 2220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:52.562366 (+ 2087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:52.564694 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:52.565701 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:52.565867 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:52.571215 (+ 5348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:52.571220 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:52.571223 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:52.571280 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:52.575507 (+ 4227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:52.576719 (+ 1212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:52.576723 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:52.576726 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:52.576923 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:52.577606 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:52.577610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:52.577612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:52.577986 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:52.578129 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:52.579150 (+ 1021us) server_negotiation.cc:299] Negotiation successful
0809 04:45:52.579329 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":959,"thread_start_us":692,"threads_started":1}
I20260809 04:45:52.581753 25686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:52.552703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44859 (local address 127.12.234.2:52935)
0809 04:45:52.554502 (+ 1799us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:52.554546 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:52.557528 (+ 2982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:52.558343 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:52.558351 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:52.559096 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:52.559982 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:52.560027 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:52.562561 (+ 2534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:52.562565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:52.564534 (+ 1969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:52.564547 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:52.566041 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:52.566047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:52.566307 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:52.567264 (+ 957us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:52.567288 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:52.570643 (+ 3355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:52.575668 (+ 5025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:52.575680 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:52.575701 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:52.576576 (+ 875us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:52.577042 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:52.577050 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:52.577056 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:52.577487 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:52.578133 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:52.578145 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:52.578668 (+ 523us) client_negotiation.cc:769] Sending connection context
0809 04:45:52.579031 (+ 363us) client_negotiation.cc:240] Negotiation successful
0809 04:45:52.579796 (+ 765us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1463,"thread_start_us":816,"threads_started":1}
I20260809 04:45:52.583842 25683 heartbeater.cc:347] Connected to a master server at 127.12.234.62:44859
I20260809 04:45:52.584324 25683 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:52.585975 25683 heartbeater.cc:511] Master 127.12.234.62:44859 requested a full tablet report, sending...
I20260809 04:45:52.588616 25288 ts_manager.cc:194] Registered new tserver with Master: 2d54543f1f3443bbb10c25b74d9de956 (127.12.234.2:46615)
I20260809 04:45:52.590430 25288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.2@KRBTEST.COM'} at 127.12.234.2:52935
WARNING: no policy specified for kudu/127.12.234.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
I20260809 04:45:52.632262 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.234.20:54093/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:0
--local_ip_for_outbound_sockets=127.12.234.3
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44859
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:33283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:45:53.324174 25689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:45:53.324527 25689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:45:53.324689 25689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:45:53.336478 25689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:45:53.336634 25689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:45:53.336789 25689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:45:53.348488 25689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:33283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.12.234.20:54093/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44859
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:45:53.349923 25689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:45:53.351289 25689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:45:53.366125 25694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:53.366397 25695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:45:53.366650 25697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:45:53.367743 25689 server_base.cc:1034] running on GCE node
I20260809 04:45:53.368438 25689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:45:53.369570 25689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:45:53.370812 25689 hybrid_clock.cc:648] HybridClock initialized: now 1786250753370740 us; error 49 us; skew 500 ppm
I20260809 04:45:53.371133 25689 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:45:53 dist-test-slave-1jjb krb5kdc[24761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250753, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:45:53.374713 25689 init.cc:377] Logged in from keytab as kudu/127.12.234.3@KRBTEST.COM (short username kudu)
I20260809 04:45:53.376289 25689 webserver.cc:460] Webserver started at http://127.12.234.3:36981/ using document root <none> and password file <none>
I20260809 04:45:53.377034 25689 fs_manager.cc:359] Metadata directory not provided
I20260809 04:45:53.377202 25689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:45:53.377558 25689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:45:53 dist-test-slave-1jjb krb5kdc[24761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250753, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for HTTP/127.12.234.20@KRBTEST.COM
I20260809 04:45:53.393929 25689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-2/data/instance:
uuid: "aa91f7a0e48f4bec86f1c3eda3976eb7"
format_stamp: "Formatted at 2026-08-09 04:45:53 on dist-test-slave-1jjb"
server_key: "8732f6ee069d675316a63bd7b034967d"
server_key_iv: "e6514f80ca2026241b90cb60281d756c"
server_key_version: "kuduclusterkey@0"
I20260809 04:45:53.394855 25689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance:
uuid: "aa91f7a0e48f4bec86f1c3eda3976eb7"
format_stamp: "Formatted at 2026-08-09 04:45:53 on dist-test-slave-1jjb"
server_key: "8732f6ee069d675316a63bd7b034967d"
server_key_iv: "e6514f80ca2026241b90cb60281d756c"
server_key_version: "kuduclusterkey@0"
I20260809 04:45:53.401284 25689 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.002s
I20260809 04:45:53.419387 25705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:53.421108 25689 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260809 04:45:53.421351 25689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "aa91f7a0e48f4bec86f1c3eda3976eb7"
format_stamp: "Formatted at 2026-08-09 04:45:53 on dist-test-slave-1jjb"
server_key: "8732f6ee069d675316a63bd7b034967d"
server_key_iv: "e6514f80ca2026241b90cb60281d756c"
server_key_version: "kuduclusterkey@0"
I20260809 04:45:53.421696 25689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:45:53.471639 25689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:45:53.477648 25689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:45:53.478199 25689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:45:53.479941 25689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:45:53.482060 25689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:45:53.482153 25689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:53.482280 25689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:45:53.482344 25689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:45:53.542779 25689 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:43793
I20260809 04:45:53.542830 25827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:43793 every 8 connection(s)
I20260809 04:45:53.544929 25689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:45:53.552485 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 25689
I20260809 04:45:53.552683 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance
I20260809 04:45:53.553273 13224 external_mini_cluster.cc:1428] Setting key ad18dcc42cb74d793c8c11fd9a1ebc57
Aug 09 04:45:53 dist-test-slave-1jjb krb5kdc[24761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250753, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:45:53.569295 25831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:53.548189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:41771 (local address 127.12.234.62:44859)
0809 04:45:53.548983 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:53.548988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:53.550240 (+ 1252us) server_negotiation.cc:407] Connection header received
0809 04:45:53.552489 (+ 2249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:53.552494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:53.552599 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:53.552736 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:53.554396 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.555852 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.557774 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.558306 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.558425 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:53.562332 (+ 3907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:53.562346 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:53.562350 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:53.562395 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:53.565713 (+ 3318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:53.566456 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:53.566460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:53.566462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:53.566612 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:53.567146 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:53.567151 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:53.567154 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:53.567575 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:53.567748 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:53.568628 (+ 880us) server_negotiation.cc:299] Negotiation successful
0809 04:45:53.568872 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":483,"threads_started":1}
I20260809 04:45:53.570298 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:53.548922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44859 (local address 127.12.234.3:41771)
0809 04:45:53.550072 (+ 1150us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:53.550117 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:53.552200 (+ 2083us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:53.553010 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:53.553018 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:53.553545 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:45:53.554191 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:53.554209 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.556014 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.556020 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:53.557623 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:53.557647 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.558572 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.558577 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:53.558792 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:53.559644 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:45:53.559667 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:45:53.562105 (+ 2438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:45:53.565853 (+ 3748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:53.565860 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:53.565871 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:53.566344 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:53.566749 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:45:53.566753 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:45:53.566755 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:45:53.567042 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:45:53.567708 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:45:53.567713 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:45:53.568210 (+ 497us) client_negotiation.cc:769] Sending connection context
0809 04:45:53.568495 (+ 285us) client_negotiation.cc:240] Negotiation successful
0809 04:45:53.569013 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":877,"thread_start_us":576,"threads_started":1}
I20260809 04:45:53.571626 25828 heartbeater.cc:347] Connected to a master server at 127.12.234.62:44859
I20260809 04:45:53.571947 25828 heartbeater.cc:464] Registering TS with master...
I20260809 04:45:53.572744 25828 heartbeater.cc:511] Master 127.12.234.62:44859 requested a full tablet report, sending...
I20260809 04:45:53.574278 25288 ts_manager.cc:194] Registered new tserver with Master: aa91f7a0e48f4bec86f1c3eda3976eb7 (127.12.234.3:43793)
I20260809 04:45:53.575258 25288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.3@KRBTEST.COM'} at 127.12.234.3:41771
I20260809 04:45:53.581697 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:45:53.593849 25683 heartbeater.cc:503] Master 127.12.234.62:44859 was elected leader, sending a full tablet report...
I20260809 04:45:53.602046 25831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:53.587008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55554 (local address 127.12.234.62:44859)
0809 04:45:53.587216 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:53.587225 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:53.587684 (+ 459us) server_negotiation.cc:407] Connection header received
0809 04:45:53.587932 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:53.587937 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:53.588012 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:53.588156 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:53.589932 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.590751 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.592357 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.592866 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.593010 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:53.596219 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:53.596224 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:53.596227 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:53.596262 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:53.599041 (+ 2779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:53.599757 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:53.599764 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:53.599768 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:53.599911 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:53.600411 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:53.600417 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:53.600420 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:53.600731 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:53.601006 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:53.601390 (+ 384us) server_negotiation.cc:299] Negotiation successful
0809 04:45:53.601652 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 04:45:53.606457 25288 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:45:53.616153 25288 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:45:53.663578 25845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:53.632500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60654 (local address 127.12.234.2:46615)
0809 04:45:53.653172 (+ 20672us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:53.653183 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:53.653210 (+ 27us) server_negotiation.cc:407] Connection header received
0809 04:45:53.653309 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:53.653314 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:53.653468 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:53.653714 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:53.655057 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.656689 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.659974 (+ 3285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.661638 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.661792 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:53.662094 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:53.662916 (+ 822us) server_negotiation.cc:299] Negotiation successful
0809 04:45:53.663095 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20511,"thread_start_us":20292,"threads_started":1}
I20260809 04:45:53.663976 25846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:53.632712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57522 (local address 127.12.234.1:36689)
0809 04:45:53.650193 (+ 17481us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:53.650202 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:53.650233 (+ 31us) server_negotiation.cc:407] Connection header received
0809 04:45:53.650355 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:53.650363 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:53.650697 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:53.651291 (+ 594us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:53.655151 (+ 3860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.657330 (+ 2179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.660900 (+ 3570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.662423 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.662606 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:53.662933 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:53.663240 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 04:45:53.663432 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17206,"thread_start_us":14315,"threads_started":1}
I20260809 04:45:53.664184 25843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:53.632174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:36689 (local address 127.0.0.1:57522)
0809 04:45:53.648553 (+ 16379us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:53.648617 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:53.648816 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:53.654114 (+ 5298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:53.654119 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:53.654149 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:53.654987 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:53.655005 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.657478 (+ 2473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.657486 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:53.660686 (+ 3200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:53.660702 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.662767 (+ 2065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.662777 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:53.663042 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:53.663067 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 04:45:53.663176 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 04:45:53.663432 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15883,"mutex_wait_us":46,"thread_start_us":16215,"threads_started":1}
I20260809 04:45:53.665261 25847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:53.632362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:46615 (local address 127.0.0.1:60654)
0809 04:45:53.651232 (+ 18870us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:53.651260 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:53.651469 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:53.653714 (+ 2245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:53.653722 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:53.653760 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:53.654746 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:53.654770 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.656934 (+ 2164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.656939 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:53.659815 (+ 2876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:53.659828 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.662306 (+ 2478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.662318 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:53.662575 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:53.662645 (+ 70us) client_negotiation.cc:769] Sending connection context
0809 04:45:53.664467 (+ 1822us) client_negotiation.cc:240] Negotiation successful
0809 04:45:53.664623 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18661,"mutex_wait_us":161,"thread_start_us":19690,"threads_started":1}
I20260809 04:45:53.668676 25450 tablet_service.cc:1467] Processing CreateTablet for tablet aa8aa34b76764e9f829de551bf40abd3 (DEFAULT_TABLE table=test-table [id=914b3d838c264a7db84afb328b5a9b23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:53.668807 25609 tablet_service.cc:1467] Processing CreateTablet for tablet aa8aa34b76764e9f829de551bf40abd3 (DEFAULT_TABLE table=test-table [id=914b3d838c264a7db84afb328b5a9b23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:53.670454 25450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa8aa34b76764e9f829de551bf40abd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:53.670514 25609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa8aa34b76764e9f829de551bf40abd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:53.671056 25844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:53.649494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:43793 (local address 127.0.0.1:45332)
0809 04:45:53.652465 (+ 2971us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:53.652501 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:53.652649 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:53.656653 (+ 4004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:53.656657 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:53.656685 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:53.657931 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:53.657946 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.661086 (+ 3140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.661091 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:53.663952 (+ 2861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:53.663968 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.669934 (+ 5966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.669942 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:53.670194 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:53.670235 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 04:45:53.670322 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 04:45:53.670521 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2778,"mutex_wait_us":43,"thread_start_us":1370,"threads_started":1}
I20260809 04:45:53.672156 25848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:53.650197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45332 (local address 127.12.234.3:43793)
0809 04:45:53.655896 (+ 5699us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:53.655906 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:53.655962 (+ 56us) server_negotiation.cc:407] Connection header received
0809 04:45:53.656113 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:53.656122 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:53.656388 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:53.656634 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:53.658215 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.660505 (+ 2290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.664186 (+ 3681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.666067 (+ 1881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.666196 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:53.671388 (+ 5192us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:53.671470 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 04:45:53.671682 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5472,"thread_start_us":5215,"threads_started":1}
I20260809 04:45:53.676604 25759 tablet_service.cc:1467] Processing CreateTablet for tablet aa8aa34b76764e9f829de551bf40abd3 (DEFAULT_TABLE table=test-table [id=914b3d838c264a7db84afb328b5a9b23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:45:53.678597 25759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa8aa34b76764e9f829de551bf40abd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:45:53.684217 25849 tablet_bootstrap.cc:492] T aa8aa34b76764e9f829de551bf40abd3 P 2d54543f1f3443bbb10c25b74d9de956: Bootstrap starting.
I20260809 04:45:53.688172 25849 tablet_bootstrap.cc:654] T aa8aa34b76764e9f829de551bf40abd3 P 2d54543f1f3443bbb10c25b74d9de956: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:53.688546 25850 tablet_bootstrap.cc:492] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767: Bootstrap starting.
I20260809 04:45:53.689954 25849 log.cc:824] T aa8aa34b76764e9f829de551bf40abd3 P 2d54543f1f3443bbb10c25b74d9de956: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:53.692947 25850 tablet_bootstrap.cc:654] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:53.694135 25849 tablet_bootstrap.cc:492] T aa8aa34b76764e9f829de551bf40abd3 P 2d54543f1f3443bbb10c25b74d9de956: No bootstrap required, opened a new log
I20260809 04:45:53.694523 25849 ts_tablet_manager.cc:1397] T aa8aa34b76764e9f829de551bf40abd3 P 2d54543f1f3443bbb10c25b74d9de956: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260809 04:45:53.694734 25850 log.cc:824] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:53.698668 25850 tablet_bootstrap.cc:492] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767: No bootstrap required, opened a new log
I20260809 04:45:53.698884 25853 tablet_bootstrap.cc:492] T aa8aa34b76764e9f829de551bf40abd3 P aa91f7a0e48f4bec86f1c3eda3976eb7: Bootstrap starting.
I20260809 04:45:53.698992 25850 ts_tablet_manager.cc:1397] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260809 04:45:53.702369 25853 tablet_bootstrap.cc:654] T aa8aa34b76764e9f829de551bf40abd3 P aa91f7a0e48f4bec86f1c3eda3976eb7: Neither blocks nor log segments found. Creating new log.
I20260809 04:45:53.703626 25853 log.cc:824] T aa8aa34b76764e9f829de551bf40abd3 P aa91f7a0e48f4bec86f1c3eda3976eb7: Log is configured to *not* fsync() on all Append() calls
I20260809 04:45:53.706727 25853 tablet_bootstrap.cc:492] T aa8aa34b76764e9f829de551bf40abd3 P aa91f7a0e48f4bec86f1c3eda3976eb7: No bootstrap required, opened a new log
I20260809 04:45:53.707024 25853 ts_tablet_manager.cc:1397] T aa8aa34b76764e9f829de551bf40abd3 P aa91f7a0e48f4bec86f1c3eda3976eb7: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.006s
I20260809 04:45:53.707767 25849 raft_consensus.cc:348] T aa8aa34b76764e9f829de551bf40abd3 P 2d54543f1f3443bbb10c25b74d9de956 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d54543f1f3443bbb10c25b74d9de956" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46615 } } peers { permanent_uuid: "5df9343ac86748e588ab079c662bb767" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36689 } } peers { permanent_uuid: "aa91f7a0e48f4bec86f1c3eda3976eb7" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 43793 } }
I20260809 04:45:53.708400 25849 raft_consensus.cc:374] T aa8aa34b76764e9f829de551bf40abd3 P 2d54543f1f3443bbb10c25b74d9de956 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:53.708549 25849 raft_consensus.cc:729] T aa8aa34b76764e9f829de551bf40abd3 P 2d54543f1f3443bbb10c25b74d9de956 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d54543f1f3443bbb10c25b74d9de956, State: Initialized, Role: FOLLOWER
I20260809 04:45:53.709280 25849 consensus_queue.cc:260] T aa8aa34b76764e9f829de551bf40abd3 P 2d54543f1f3443bbb10c25b74d9de956 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d54543f1f3443bbb10c25b74d9de956" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46615 } } peers { permanent_uuid: "5df9343ac86748e588ab079c662bb767" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36689 } } peers { permanent_uuid: "aa91f7a0e48f4bec86f1c3eda3976eb7" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 43793 } }
I20260809 04:45:53.711565 25849 ts_tablet_manager.cc:1428] T aa8aa34b76764e9f829de551bf40abd3 P 2d54543f1f3443bbb10c25b74d9de956: Time spent starting tablet: real 0.017s user 0.015s sys 0.004s
I20260809 04:45:53.713265 25850 raft_consensus.cc:348] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d54543f1f3443bbb10c25b74d9de956" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46615 } } peers { permanent_uuid: "5df9343ac86748e588ab079c662bb767" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36689 } } peers { permanent_uuid: "aa91f7a0e48f4bec86f1c3eda3976eb7" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 43793 } }
I20260809 04:45:53.713680 25850 raft_consensus.cc:374] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:53.713819 25850 raft_consensus.cc:729] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5df9343ac86748e588ab079c662bb767, State: Initialized, Role: FOLLOWER
I20260809 04:45:53.714453 25850 consensus_queue.cc:260] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d54543f1f3443bbb10c25b74d9de956" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46615 } } peers { permanent_uuid: "5df9343ac86748e588ab079c662bb767" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36689 } } peers { permanent_uuid: "aa91f7a0e48f4bec86f1c3eda3976eb7" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 43793 } }
I20260809 04:45:53.716562 25850 ts_tablet_manager.cc:1428] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767: Time spent starting tablet: real 0.017s user 0.011s sys 0.005s
I20260809 04:45:53.719924 25853 raft_consensus.cc:348] T aa8aa34b76764e9f829de551bf40abd3 P aa91f7a0e48f4bec86f1c3eda3976eb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d54543f1f3443bbb10c25b74d9de956" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46615 } } peers { permanent_uuid: "5df9343ac86748e588ab079c662bb767" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36689 } } peers { permanent_uuid: "aa91f7a0e48f4bec86f1c3eda3976eb7" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 43793 } }
I20260809 04:45:53.720438 25853 raft_consensus.cc:374] T aa8aa34b76764e9f829de551bf40abd3 P aa91f7a0e48f4bec86f1c3eda3976eb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:45:53.720566 25853 raft_consensus.cc:729] T aa8aa34b76764e9f829de551bf40abd3 P aa91f7a0e48f4bec86f1c3eda3976eb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa91f7a0e48f4bec86f1c3eda3976eb7, State: Initialized, Role: FOLLOWER
I20260809 04:45:53.721117 25853 consensus_queue.cc:260] T aa8aa34b76764e9f829de551bf40abd3 P aa91f7a0e48f4bec86f1c3eda3976eb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d54543f1f3443bbb10c25b74d9de956" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46615 } } peers { permanent_uuid: "5df9343ac86748e588ab079c662bb767" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36689 } } peers { permanent_uuid: "aa91f7a0e48f4bec86f1c3eda3976eb7" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 43793 } }
I20260809 04:45:53.722302 25828 heartbeater.cc:503] Master 127.12.234.62:44859 was elected leader, sending a full tablet report...
I20260809 04:45:53.722692 25853 ts_tablet_manager.cc:1428] T aa8aa34b76764e9f829de551bf40abd3 P aa91f7a0e48f4bec86f1c3eda3976eb7: Time spent starting tablet: real 0.016s user 0.013s sys 0.003s
W20260809 04:45:53.797307 25830 tablet.cc:2369] T aa8aa34b76764e9f829de551bf40abd3 P aa91f7a0e48f4bec86f1c3eda3976eb7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:45:53.799136 25857 raft_consensus.cc:482] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:45:53.799643 25857 raft_consensus.cc:504] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d54543f1f3443bbb10c25b74d9de956" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46615 } } peers { permanent_uuid: "5df9343ac86748e588ab079c662bb767" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36689 } } peers { permanent_uuid: "aa91f7a0e48f4bec86f1c3eda3976eb7" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 43793 } }
W20260809 04:45:53.801049 25684 tablet.cc:2369] T aa8aa34b76764e9f829de551bf40abd3 P 2d54543f1f3443bbb10c25b74d9de956: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:45:53.801963 25857 leader_election.cc:288] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d54543f1f3443bbb10c25b74d9de956 (127.12.234.2:46615), aa91f7a0e48f4bec86f1c3eda3976eb7 (127.12.234.3:43793)
I20260809 04:45:53.811813 25848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:53.802335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:33481 (local address 127.12.234.3:43793)
0809 04:45:53.802594 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:53.802603 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:53.804331 (+ 1728us) server_negotiation.cc:407] Connection header received
0809 04:45:53.804520 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:53.804524 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:53.804645 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:53.804755 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:53.805743 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.806499 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.809501 (+ 3002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.810639 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.810785 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:53.810840 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:53.811366 (+ 526us) server_negotiation.cc:299] Negotiation successful
0809 04:45:53.811503 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 04:45:53.812186 25845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:53.802518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:46773 (local address 127.12.234.2:46615)
0809 04:45:53.802808 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:53.802817 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:53.804145 (+ 1328us) server_negotiation.cc:407] Connection header received
0809 04:45:53.804369 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:53.804380 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:53.804475 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:53.804578 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:45:53.805459 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.806586 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.809643 (+ 3057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.811020 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.811194 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:53.811270 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:53.811651 (+ 381us) server_negotiation.cc:299] Negotiation successful
0809 04:45:53.811806 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 04:45:53.812325 25860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:53.802578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:46615 (local address 127.12.234.1:46773)
0809 04:45:53.803976 (+ 1398us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:53.803992 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:53.804198 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:53.804620 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:53.804625 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:53.804652 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:53.805290 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:53.805300 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.806729 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.806732 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:53.809388 (+ 2656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:53.809408 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.811256 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.811261 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:53.811451 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:53.811484 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 04:45:53.811586 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 04:45:53.811717 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1244,"thread_start_us":970,"threads_started":1}
I20260809 04:45:53.812395 25859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:53.802642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:43793 (local address 127.12.234.1:33481)
0809 04:45:53.804175 (+ 1533us) negotiation.cc:107] Waiting for socket to connect
0809 04:45:53.804196 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:45:53.804389 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:45:53.804799 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:45:53.804811 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:45:53.804856 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:45:53.805538 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:53.805555 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.806682 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.806692 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:53.809280 (+ 2588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:45:53.809294 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.810917 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.810925 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:45:53.811167 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:53.811212 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 04:45:53.811322 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 04:45:53.811451 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1306,"mutex_wait_us":99,"spinlock_wait_cycles":2432,"thread_start_us":905,"threads_started":1}
I20260809 04:45:53.813414 25781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa8aa34b76764e9f829de551bf40abd3" candidate_uuid: "5df9343ac86748e588ab079c662bb767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa91f7a0e48f4bec86f1c3eda3976eb7" is_pre_election: true
I20260809 04:45:53.813993 25781 raft_consensus.cc:2454] T aa8aa34b76764e9f829de551bf40abd3 P aa91f7a0e48f4bec86f1c3eda3976eb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5df9343ac86748e588ab079c662bb767 in term 0.
I20260809 04:45:53.813874 25631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa8aa34b76764e9f829de551bf40abd3" candidate_uuid: "5df9343ac86748e588ab079c662bb767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d54543f1f3443bbb10c25b74d9de956" is_pre_election: true
I20260809 04:45:53.814464 25631 raft_consensus.cc:2454] T aa8aa34b76764e9f829de551bf40abd3 P 2d54543f1f3443bbb10c25b74d9de956 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5df9343ac86748e588ab079c662bb767 in term 0.
I20260809 04:45:53.814878 25400 leader_election.cc:302] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5df9343ac86748e588ab079c662bb767, aa91f7a0e48f4bec86f1c3eda3976eb7; no voters:
I20260809 04:45:53.815411 25857 raft_consensus.cc:2781] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:45:53.815624 25857 raft_consensus.cc:482] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:45:53.815757 25857 raft_consensus.cc:3037] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:53.818722 25857 raft_consensus.cc:504] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d54543f1f3443bbb10c25b74d9de956" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46615 } } peers { permanent_uuid: "5df9343ac86748e588ab079c662bb767" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36689 } } peers { permanent_uuid: "aa91f7a0e48f4bec86f1c3eda3976eb7" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 43793 } }
I20260809 04:45:53.819525 25857 leader_election.cc:288] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767 [CANDIDATE]: Term 1 election: Requested vote from peers 2d54543f1f3443bbb10c25b74d9de956 (127.12.234.2:46615), aa91f7a0e48f4bec86f1c3eda3976eb7 (127.12.234.3:43793)
I20260809 04:45:53.819941 25631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa8aa34b76764e9f829de551bf40abd3" candidate_uuid: "5df9343ac86748e588ab079c662bb767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d54543f1f3443bbb10c25b74d9de956"
I20260809 04:45:53.820191 25781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa8aa34b76764e9f829de551bf40abd3" candidate_uuid: "5df9343ac86748e588ab079c662bb767" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa91f7a0e48f4bec86f1c3eda3976eb7"
I20260809 04:45:53.820236 25631 raft_consensus.cc:3037] T aa8aa34b76764e9f829de551bf40abd3 P 2d54543f1f3443bbb10c25b74d9de956 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:53.820456 25781 raft_consensus.cc:3037] T aa8aa34b76764e9f829de551bf40abd3 P aa91f7a0e48f4bec86f1c3eda3976eb7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:45:53.824707 25631 raft_consensus.cc:2454] T aa8aa34b76764e9f829de551bf40abd3 P 2d54543f1f3443bbb10c25b74d9de956 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5df9343ac86748e588ab079c662bb767 in term 1.
I20260809 04:45:53.824822 25781 raft_consensus.cc:2454] T aa8aa34b76764e9f829de551bf40abd3 P aa91f7a0e48f4bec86f1c3eda3976eb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5df9343ac86748e588ab079c662bb767 in term 1.
I20260809 04:45:53.825250 25401 leader_election.cc:302] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d54543f1f3443bbb10c25b74d9de956, 5df9343ac86748e588ab079c662bb767; no voters:
I20260809 04:45:53.825730 25857 raft_consensus.cc:2781] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:45:53.826313 25857 raft_consensus.cc:686] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767 [term 1 LEADER]: Becoming Leader. State: Replica: 5df9343ac86748e588ab079c662bb767, State: Running, Role: LEADER
I20260809 04:45:53.827004 25857 consensus_queue.cc:237] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d54543f1f3443bbb10c25b74d9de956" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46615 } } peers { permanent_uuid: "5df9343ac86748e588ab079c662bb767" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36689 } } peers { permanent_uuid: "aa91f7a0e48f4bec86f1c3eda3976eb7" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 43793 } }
I20260809 04:45:53.832348 25288 catalog_manager.cc:5495] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5df9343ac86748e588ab079c662bb767 (127.12.234.1). New cstate: current_term: 1 leader_uuid: "5df9343ac86748e588ab079c662bb767" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d54543f1f3443bbb10c25b74d9de956" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5df9343ac86748e588ab079c662bb767" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa91f7a0e48f4bec86f1c3eda3976eb7" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 43793 } health_report { overall_health: UNKNOWN } } }
W20260809 04:45:53.868414 25524 tablet.cc:2369] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:45:53.881596 25846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:53.875296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57526 (local address 127.12.234.1:36689)
0809 04:45:53.875546 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:53.875551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:53.875572 (+ 21us) server_negotiation.cc:407] Connection header received
0809 04:45:53.875730 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:53.875734 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:53.875832 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:53.875979 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:45:53.876443 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.877322 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.879036 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.879599 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.879706 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:53.880324 (+ 618us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:45:53.880867 (+ 543us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:45:53.880988 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:53.881121 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 04:45:53.881205 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 04:45:53.891177 25631 raft_consensus.cc:1261] T aa8aa34b76764e9f829de551bf40abd3 P 2d54543f1f3443bbb10c25b74d9de956 [term 1 FOLLOWER]: Refusing update from remote peer 5df9343ac86748e588ab079c662bb767: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:53.891199 25781 raft_consensus.cc:1261] T aa8aa34b76764e9f829de551bf40abd3 P aa91f7a0e48f4bec86f1c3eda3976eb7 [term 1 FOLLOWER]: Refusing update from remote peer 5df9343ac86748e588ab079c662bb767: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:45:53.892271 25857 consensus_queue.cc:1035] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d54543f1f3443bbb10c25b74d9de956" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:53.892649 25861 consensus_queue.cc:1035] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa91f7a0e48f4bec86f1c3eda3976eb7" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 43793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:45:53.907065 25864 mvcc.cc:205] Tried to move back new op lower bound from 7316483087927767040 to 7316483087680311296. Current Snapshot: MvccSnapshot[applied={T|T < 7316483087927767040}]
I20260809 04:45:53.908900 25865 mvcc.cc:205] Tried to move back new op lower bound from 7316483087927767040 to 7316483087680311296. Current Snapshot: MvccSnapshot[applied={T|T < 7316483087927767040}]
I20260809 04:45:53.913590 25866 mvcc.cc:205] Tried to move back new op lower bound from 7316483087927767040 to 7316483087680311296. Current Snapshot: MvccSnapshot[applied={T|T < 7316483087927767040}]
I20260809 04:45:53.927872 25288 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:53.928188 25288 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:45:53.932420 25288 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42d0b6e8cb584a3b841961e050f24643: Sending DeleteTablet for 3 replicas of tablet aa8aa34b76764e9f829de551bf40abd3
I20260809 04:45:53.933880 25609 tablet_service.cc:1514] Processing DeleteTablet for tablet aa8aa34b76764e9f829de551bf40abd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:53 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:60654
I20260809 04:45:53.933853 25450 tablet_service.cc:1514] Processing DeleteTablet for tablet aa8aa34b76764e9f829de551bf40abd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:53 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:57522
I20260809 04:45:53.934312 25759 tablet_service.cc:1514] Processing DeleteTablet for tablet aa8aa34b76764e9f829de551bf40abd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:45:53 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:45332
I20260809 04:45:53.936770 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 25376
I20260809 04:45:53.946729 25873 tablet_replica.cc:330] T aa8aa34b76764e9f829de551bf40abd3 P 2d54543f1f3443bbb10c25b74d9de956: stopping tablet replica
I20260809 04:45:53.947463 25873 raft_consensus.cc:2229] T aa8aa34b76764e9f829de551bf40abd3 P 2d54543f1f3443bbb10c25b74d9de956 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:53.948259 25873 raft_consensus.cc:2258] T aa8aa34b76764e9f829de551bf40abd3 P 2d54543f1f3443bbb10c25b74d9de956 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:53.950826 25873 ts_tablet_manager.cc:1905] T aa8aa34b76764e9f829de551bf40abd3 P 2d54543f1f3443bbb10c25b74d9de956: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:53.961596 25873 ts_tablet_manager.cc:1918] T aa8aa34b76764e9f829de551bf40abd3 P 2d54543f1f3443bbb10c25b74d9de956: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:45:53.962136 25873 log.cc:1196] T aa8aa34b76764e9f829de551bf40abd3 P 2d54543f1f3443bbb10c25b74d9de956: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-1/wal/wals/aa8aa34b76764e9f829de551bf40abd3
I20260809 04:45:53.963069 25873 ts_tablet_manager.cc:1939] T aa8aa34b76764e9f829de551bf40abd3 P 2d54543f1f3443bbb10c25b74d9de956: Deleting consensus metadata
I20260809 04:45:53.965457 25271 catalog_manager.cc:4841] TS 2d54543f1f3443bbb10c25b74d9de956 (127.12.234.2:46615): tablet aa8aa34b76764e9f829de551bf40abd3 (table test-table [id=914b3d838c264a7db84afb328b5a9b23]) successfully deleted
I20260809 04:45:53.967856 25875 tablet_replica.cc:330] T aa8aa34b76764e9f829de551bf40abd3 P aa91f7a0e48f4bec86f1c3eda3976eb7: stopping tablet replica
I20260809 04:45:53.968825 25874 tablet_replica.cc:330] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767: stopping tablet replica
I20260809 04:45:53.968851 25875 raft_consensus.cc:2229] T aa8aa34b76764e9f829de551bf40abd3 P aa91f7a0e48f4bec86f1c3eda3976eb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:45:53.969394 25874 raft_consensus.cc:2229] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:45:53.969506 25875 raft_consensus.cc:2258] T aa8aa34b76764e9f829de551bf40abd3 P aa91f7a0e48f4bec86f1c3eda3976eb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:53.970909 25874 raft_consensus.cc:2258] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:45:53.972035 25875 ts_tablet_manager.cc:1905] T aa8aa34b76764e9f829de551bf40abd3 P aa91f7a0e48f4bec86f1c3eda3976eb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:45:53.973707 25874 ts_tablet_manager.cc:1905] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 04:45:53 dist-test-slave-1jjb krb5kdc[24761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.1@KRBTEST.COM
I20260809 04:45:53.979652 25875 ts_tablet_manager.cc:1918] T aa8aa34b76764e9f829de551bf40abd3 P aa91f7a0e48f4bec86f1c3eda3976eb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:45:53.979822 25875 log.cc:1196] T aa8aa34b76764e9f829de551bf40abd3 P aa91f7a0e48f4bec86f1c3eda3976eb7: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-2/wal/wals/aa8aa34b76764e9f829de551bf40abd3
I20260809 04:45:53.980351 25875 ts_tablet_manager.cc:1939] T aa8aa34b76764e9f829de551bf40abd3 P aa91f7a0e48f4bec86f1c3eda3976eb7: Deleting consensus metadata
I20260809 04:45:53.982465 25270 catalog_manager.cc:4841] TS aa91f7a0e48f4bec86f1c3eda3976eb7 (127.12.234.3:43793): tablet aa8aa34b76764e9f829de551bf40abd3 (table test-table [id=914b3d838c264a7db84afb328b5a9b23]) successfully deleted
I20260809 04:45:53.983177 25874 ts_tablet_manager.cc:1918] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:45:53.983441 25874 log.cc:1196] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786250603326224-13224-0/minicluster-data/ts-0/wal/wals/aa8aa34b76764e9f829de551bf40abd3
I20260809 04:45:53.984218 25874 ts_tablet_manager.cc:1939] T aa8aa34b76764e9f829de551bf40abd3 P 5df9343ac86748e588ab079c662bb767: Deleting consensus metadata
I20260809 04:45:53.986164 25270 catalog_manager.cc:4841] TS 5df9343ac86748e588ab079c662bb767 (127.12.234.1:36689): tablet aa8aa34b76764e9f829de551bf40abd3 (table test-table [id=914b3d838c264a7db84afb328b5a9b23]) successfully deleted
I20260809 04:45:53.986501 25846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:53.938672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57532 (local address 127.12.234.1:36689)
0809 04:45:53.938913 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:53.938922 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:53.963203 (+ 24281us) server_negotiation.cc:407] Connection header received
0809 04:45:53.963801 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:53.963808 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:53.963934 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:53.964014 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:53.966590 (+ 2576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.968213 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.970564 (+ 2351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:53.971527 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:53.971714 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:53.977820 (+ 6106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:53.977834 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:53.977853 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:53.977923 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:53.982811 (+ 4888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:53.983680 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:53.983689 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:53.983692 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:53.983864 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:53.984359 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:53.984364 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:53.984367 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:53.984722 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:53.985043 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:53.985396 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 04:45:53.985672 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 04:45:53.987432 25517 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:45:54.089592 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 25376
I20260809 04:45:54.111289 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 25531
Aug 09 04:45:54 dist-test-slave-1jjb krb5kdc[24761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.2@KRBTEST.COM
I20260809 04:45:54.127338 25845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:54.112194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60662 (local address 127.12.234.2:46615)
0809 04:45:54.112399 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:54.112408 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:54.112429 (+ 21us) server_negotiation.cc:407] Connection header received
0809 04:45:54.112666 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:54.112672 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:54.112753 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:54.112858 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:54.114205 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:54.114943 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:54.116418 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:54.116848 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:54.116986 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:54.121305 (+ 4319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:54.121318 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:54.121335 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:54.121368 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:54.124234 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:54.124918 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:54.124925 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:54.124928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:54.125067 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:54.125596 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:54.125600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:54.125602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:54.125983 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:54.126263 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:54.126748 (+ 485us) server_negotiation.cc:299] Negotiation successful
0809 04:45:54.126933 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 04:45:54.128201 25678 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:45:54.229002 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 25531
I20260809 04:45:54.258334 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 25689
Aug 09 04:45:54 dist-test-slave-1jjb krb5kdc[24761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.3@KRBTEST.COM
I20260809 04:45:54.275130 25848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:45:54.259247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45348 (local address 127.12.234.3:43793)
0809 04:45:54.259429 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 04:45:54.259436 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:45:54.259461 (+ 25us) server_negotiation.cc:407] Connection header received
0809 04:45:54.259701 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:45:54.259705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:45:54.259777 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:45:54.259875 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:45:54.261788 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:54.262510 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:54.263851 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:45:54.264347 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:45:54.264433 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:45:54.269388 (+ 4955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:45:54.269405 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:45:54.269423 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:45:54.269463 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:45:54.272182 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:54.272921 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:54.272929 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:54.272932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:54.273084 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:45:54.273482 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:45:54.273495 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:45:54.273498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:45:54.273870 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:45:54.274191 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0809 04:45:54.274528 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 04:45:54.274752 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 04:45:54.275821 25823 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:45:54.368742 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 25689
I20260809 04:45:54.391157 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 25252
I20260809 04:45:54.392433 25320 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:45:54.508562 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 25252
I20260809 04:45:54.537318 13224 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 04:45:54.881543 13224 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 04:45:54.881848 13224 mini_ranger.cc:66] Stopping Ranger...
I20260809 04:45:55.227131 13224 mini_ranger.cc:68] Stopped Ranger
2026-08-09 04:45:55.227 UTC [24795] LOG: received smart shutdown request
2026-08-09 04:45:55.231 UTC [24795] LOG: background worker "logical replication launcher" (PID 24804) exited with exit code 1
2026-08-09 04:45:55.231 UTC [24799] LOG: shutting down
2026-08-09 04:45:55.402 UTC [24795] LOG: database system is shut down
2026-08-09T04:45:55Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50536 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:45:55 dist-test-slave-1jjb krb5kdc[25892](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:45:55 dist-test-slave-1jjb krb5kdc[25892](info): set up 2 sockets
Aug 09 04:45:55 dist-test-slave-1jjb krb5kdc[25892](info): commencing operation
krb5kdc: starting...
W20260809 04:45:56.740172 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:45:56 dist-test-slave-1jjb krb5kdc[25892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:45:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:45:56Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.12.234.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.12.234.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.12.234.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/rangeradmin_127.12.234.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.12.234.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/rangeradmin_127.12.234.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.12.234.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.12.234.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.12.234.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/rangerlookup_127.12.234.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.12.234.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/rangerlookup_127.12.234.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.234.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.12.234.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.234.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/HTTP_127.12.234.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.234.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/HTTP_127.12.234.20@KRBTEST.COM.keytab.
I20260809 04:45:57.142887 13224 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskQ3A_6j/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/postgres -l logfile start
2026-08-09 04:45:59.704 UTC [25927] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 04:45:59.704 UTC [25927] LOG: listening on IPv4 address "127.12.234.20", port 37713
2026-08-09 04:45:59.708 UTC [25927] LOG: listening on Unix socket "/tmp/.s.PGSQL.37713"
2026-08-09 04:45:59.734 UTC [25930] LOG: database system was shut down at 2026-08-09 04:45:58 UTC
2026-08-09 04:45:59.738 UTC [25927] LOG: database system is ready to accept connections
I20260809 04:46:00.808881 13224 mini_postgres.cc:96] Postgres bound to 37713
2026-08-09 04:46:00.834 UTC [25940] FATAL: database "slave" does not exist
127.12.234.20:37713 - accepting connections
I20260809 04:46:00.835588 13224 mini_ranger.cc:161] Starting Ranger...
I20260809 04:46:00.867280 13224 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 04:46:01.369011 13224 mini_ranger.cc:87] Created ranger Postgres database
I20260809 04:46:01.369163 13224 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ranger-admin
2026-08-09 04:46:01,680 [I] DB FLAVOR :POSTGRES
2026-08-09 04:46:01,680 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 04:46:01,680 [I] Checking connection..
2026-08-09 04:46:01,681 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:37713/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 04:46:01,982 [I] Checking connection passed.
2026-08-09 04:46:01,982 [I] --------- Verifying version history table ---------
2026-08-09 04:46:01,982 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:37713/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 04:46:02,260 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 04:46:02,260 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:37713/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 04:46:02,503 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 04:46:02,503 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 04:46:02,503 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:37713/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 04:46:02.766 UTC [26006] WARNING: there is no transaction in progress
2026-08-09 04:46:02,773 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 04:46:02,774 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:37713/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 04:46:03,022 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 04:46:03,022 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 04:46:03,023 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:37713/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 04:46:03,268 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:37713/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 04:46:03,533 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:37713/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1jjb.c.gcp-upstream.internal','N') ;"
2026-08-09 04:46:03,768 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:37713/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 04:46:04,037 [I] Table x_portal_user does not exist in database ranger
2026-08-09 04:46:04,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:37713/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 04:46:04,309 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 04:46:04,310 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:37713/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 04:46:04,580 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:37713/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 04:46:04,817 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 04:46:04,817 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:37713/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 04:46:05.783 UTC [26130] WARNING: there is no transaction in progress
2026-08-09 04:46:05.799 UTC [26130] WARNING: there is no transaction in progress
2026-08-09 04:46:05.816 UTC [26130] WARNING: there is no transaction in progress
2026-08-09 04:46:05.831 UTC [26130] WARNING: there is no transaction in progress
2026-08-09 04:46:05.848 UTC [26130] WARNING: there is no transaction in progress
2026-08-09 04:46:05.867 UTC [26130] WARNING: there is no transaction in progress
2026-08-09 04:46:05.923 UTC [26130] WARNING: there is no transaction in progress
2026-08-09 04:46:05.931 UTC [26130] WARNING: there is no transaction in progress
2026-08-09 04:46:05.946 UTC [26130] WARNING: there is no transaction in progress
2026-08-09 04:46:05.958 UTC [26130] WARNING: there is no transaction in progress
2026-08-09 04:46:05.968 UTC [26130] WARNING: there is no transaction in progress
2026-08-09 04:46:05.982 UTC [26130] WARNING: there is no transaction in progress
2026-08-09 04:46:05.992 UTC [26130] WARNING: there is no transaction in progress
2026-08-09 04:46:05.999 UTC [26130] WARNING: there is no transaction in progress
2026-08-09 04:46:06.006 UTC [26130] WARNING: there is no transaction in progress
2026-08-09 04:46:06.013 UTC [26130] WARNING: there is no transaction in progress
2026-08-09 04:46:06,659 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 04:46:06,660 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:37713/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 04:46:06,923 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 04:46:06,923 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:37713/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 04:46:07,175 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 04:46:07,176 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:37713/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 04:46:07,442 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 04:46:07,442 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:37713/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 04:46:07,679 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:37713/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 04:46:07,940 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:37713/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1jjb.c.gcp-upstream.internal';"
2026-08-09 04:46:08,187 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 04:46:08,188 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:37713/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 04:46:08,447 [I] DB_PATCHES have already been applied
I20260809 04:46:08.454119 13224 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ranger-admin:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskQ3A_6j/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 04:46:08.454206 13224 mini_ranger.cc:193] Using host: 127.12.234.20
I20260809 04:46:08.476253 13224 mini_ranger.cc:227] Ranger admin URL: http://127.12.234.20:38307
Aug 09, 2026 4:46:08 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:46:08 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:46:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 4:46:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 4:46:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 4:46:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 4:46:09 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 04:46:09.667112 13224 mini_ranger.cc:160] Time spent starting Ranger: real 8.832s user 0.000s sys 0.063s
Aug 09, 2026 4:46:30 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 4:46:30 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Aug 09, 2026 4:46:30 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 4:46:30 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 4:46:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 4:46:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 4:46:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 4:46:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 4:46:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 4:46:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 4:46:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 4:46:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 4:46:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 4:46:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 4:46:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 4:46:30 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 4:46:30 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 4:46:31 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 04:46:32.349516 13224 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.12.234.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.12.234.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.12.234.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/rangerkms_127.12.234.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.12.234.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/rangerkms_127.12.234.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.234.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.12.234.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.234.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/HTTP_127.12.234.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.234.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/HTTP_127.12.234.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 04:46:32 dist-test-slave-1jjb krb5kdc[25892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250792, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 04:46:32.555615 13224 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 04:46:32.581720 13224 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 04:46:33.442173 13224 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 04:46:33.442308 13224 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ranger-kms
I20260809 04:46:33.442400 13224 mini_ranger_kms.cc:227] Using postgres at 127.12.234.20:37713
2026-08-09 04:46:33,850 [I] DB FLAVOR :POSTGRES
2026-08-09 04:46:33,850 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 04:46:33,850 [I] Checking connection
2026-08-09 04:46:33,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:37713/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 04:46:34,100 [I] connection success
2026-08-09 04:46:34,101 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 04:46:34,101 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 04:46:34,101 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:37713/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 04:46:34,340 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 04:46:34,341 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 04:46:34,341 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 04:46:34,341 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.234.20:37713/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 04:46:34,624 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 04:46:35.052623 13224 mini_ranger_kms.cc:310] Created kms service
I20260809 04:46:35.235970 13224 mini_ranger_kms.cc:326] Created kudu user
I20260809 04:46:35.283842 13224 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 04:46:35 dist-test-slave-1jjb krb5kdc[25892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250792, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.12.234.20@KRBTEST.COM
I20260809 04:46:35.683279 13224 mini_ranger_kms.cc:384] Added ranger policy
I20260809 04:46:35.683566 13224 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ranger-kms:/tmp/dist-test-taskQ3A_6j/build/asan/bin/postgresql.jar:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskQ3A_6j/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 04:46:35.683626 13224 mini_ranger_kms.cc:240] Using host: 127.12.234.20
I20260809 04:46:35.709460 13224 mini_ranger_kms.cc:276] Ranger KMS PID: 26336
I20260809 04:46:35.709571 13224 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.12.234.20:47151
Aug 09, 2026 4:46:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 4:46:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 4:46:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 4:46:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 4:46:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 4:46:37 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 04:46:37 dist-test-slave-1jjb krb5kdc[25892](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786250797, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.234.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:46:37.989156 13224 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.434s user 0.047s sys 0.388s
I20260809 04:46:37.989435 13224 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 04:46:37.989675 13224 mini_ranger_kms.cc:401] 127.12.234.20:47151/kms/v1/keys
Aug 09 04:46:38 dist-test-slave-1jjb krb5kdc[25892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250797, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.234.20@KRBTEST.COM for HTTP/127.12.234.20@KRBTEST.COM
Aug 09 04:46:38 dist-test-slave-1jjb krb5kdc[25892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250797, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.234.20@KRBTEST.COM for HTTP/127.12.234.20@KRBTEST.COM
Aug 09 04:46:40 dist-test-slave-1jjb krb5kdc[25892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:46:40.679878 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:36545
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:35085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:36545
--ranger_config_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.234.20:47151/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:46:41.012856 26393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:46:41.013149 26393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:46:41.013302 26393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:46:41.019995 26393 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:46:41.020160 26393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:46:41.020249 26393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:46:41.020339 26393 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:46:41.020423 26393 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:46:41.029778 26393 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:35085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/wal
--ranger_kms_url=127.12.234.20:47151/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:36545
--ranger_config_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:36545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:46:41.030946 26393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:46:41.032217 26393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:46:41.045600 26398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:41.045885 26399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:41.046139 26401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:41.046612 26393 server_base.cc:1034] running on GCE node
I20260809 04:46:41.047528 26393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:46:41.048967 26393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:46:41.050204 26393 hybrid_clock.cc:648] HybridClock initialized: now 1786250801050148 us; error 49 us; skew 500 ppm
I20260809 04:46:41.050547 26393 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:46:41 dist-test-slave-1jjb krb5kdc[25892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250801, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:46:41.059689 26393 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:46:41.061582 26393 webserver.cc:460] Webserver started at http://127.12.234.62:36335/ using document root <none> and password file <none>
I20260809 04:46:41.062486 26393 fs_manager.cc:359] Metadata directory not provided
I20260809 04:46:41.062628 26393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:46:41.062956 26393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:46:41 dist-test-slave-1jjb krb5kdc[25892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250801, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for HTTP/127.12.234.20@KRBTEST.COM
I20260809 04:46:41.117718 26393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "c5d11477f1404dce8742781cea6893c3"
format_stamp: "Formatted at 2026-08-09 04:46:41 on dist-test-slave-1jjb"
server_key: "cc312fcc15494a797764b9c2db1fffeb"
server_key_iv: "c4d331e3a3a50e918c0667238eff5011"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:41.118570 26393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "c5d11477f1404dce8742781cea6893c3"
format_stamp: "Formatted at 2026-08-09 04:46:41 on dist-test-slave-1jjb"
server_key: "cc312fcc15494a797764b9c2db1fffeb"
server_key_iv: "c4d331e3a3a50e918c0667238eff5011"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:41.123870 26393 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 04:46:41.141548 26409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:41.142961 26393 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:46:41.143157 26393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "c5d11477f1404dce8742781cea6893c3"
format_stamp: "Formatted at 2026-08-09 04:46:41 on dist-test-slave-1jjb"
server_key: "cc312fcc15494a797764b9c2db1fffeb"
server_key_iv: "c4d331e3a3a50e918c0667238eff5011"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:41.143361 26393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:46:41.161233 26393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:46:41.164929 26393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:46:41.165220 26393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:46:41.183877 26393 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:36545
I20260809 04:46:41.183954 26468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:36545 every 8 connection(s)
I20260809 04:46:41.185151 26393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:46:41.189567 26469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:46:41.193425 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 26393
I20260809 04:46:41.193634 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:46:41.194308 13224 external_mini_cluster.cc:1428] Setting key e61b05e63f6360535d4e93e8f135d5c1
I20260809 04:46:41.203130 26469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3: Bootstrap starting.
I20260809 04:46:41.206691 26469 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3: Neither blocks nor log segments found. Creating new log.
I20260809 04:46:41.208033 26469 log.cc:824] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3: Log is configured to *not* fsync() on all Append() calls
I20260809 04:46:41.211158 26469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3: No bootstrap required, opened a new log
I20260809 04:46:41.219892 26469 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d11477f1404dce8742781cea6893c3" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36545 } }
I20260809 04:46:41.220331 26469 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:46:41.220441 26469 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5d11477f1404dce8742781cea6893c3, State: Initialized, Role: FOLLOWER
I20260809 04:46:41.221134 26469 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d11477f1404dce8742781cea6893c3" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36545 } }
I20260809 04:46:41.221415 26469 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:46:41.221541 26469 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:46:41.221695 26469 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 04:46:41 dist-test-slave-1jjb krb5kdc[25892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:46:41.223757 26469 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d11477f1404dce8742781cea6893c3" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36545 } }
I20260809 04:46:41.224306 26469 leader_election.cc:302] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5d11477f1404dce8742781cea6893c3; no voters:
I20260809 04:46:41.224977 26469 leader_election.cc:288] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:46:41.225154 26475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:46:41.227010 26469 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:46:41.232160 26475 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [term 1 LEADER]: Becoming Leader. State: Replica: c5d11477f1404dce8742781cea6893c3, State: Running, Role: LEADER
I20260809 04:46:41.233726 26473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:41.196518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37490 (local address 127.12.234.62:36545)
0809 04:46:41.208613 (+ 12095us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:41.208630 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:41.209143 (+ 513us) server_negotiation.cc:407] Connection header received
0809 04:46:41.211443 (+ 2300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:41.211470 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:41.211891 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:41.212277 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:41.213822 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:41.215784 (+ 1962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:41.217637 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:41.218679 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:41.218849 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:41.223580 (+ 4731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:41.223592 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:41.223611 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:41.223659 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:41.227165 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:41.228013 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:41.228020 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:41.228028 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:41.228244 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:41.228812 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:41.228820 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:41.228826 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:41.229713 (+ 887us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:41.230068 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:41.231748 (+ 1680us) server_negotiation.cc:299] Negotiation successful
0809 04:46:41.232268 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11794,"thread_start_us":11553,"threads_started":1}
I20260809 04:46:41.232753 26475 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d11477f1404dce8742781cea6893c3" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36545 } }
I20260809 04:46:41.237231 26477 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5d11477f1404dce8742781cea6893c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d11477f1404dce8742781cea6893c3" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36545 } } }
I20260809 04:46:41.237628 26477 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [sys.catalog]: This master's current role is: LEADER
I20260809 04:46:41.238097 26476 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5d11477f1404dce8742781cea6893c3. Latest consensus state: current_term: 1 leader_uuid: "c5d11477f1404dce8742781cea6893c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d11477f1404dce8742781cea6893c3" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36545 } } }
I20260809 04:46:41.238374 26476 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [sys.catalog]: This master's current role is: LEADER
I20260809 04:46:41.246475 26482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:46:41.254878 26482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:46:41.271653 26482 catalog_manager.cc:1347] Generated new cluster ID: 7d8614f8935f43f3b55d0d99c4284cc3
I20260809 04:46:41.271754 26482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:46:41.276114 26469 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 04:46:41.304008 26482 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:46:41.307157 26482 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:46:41.330513 26482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3: Generated new TSK 0
I20260809 04:46:41.331575 26482 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
I20260809 04:46:42.099517 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.234.20:47151/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:0
--local_ip_for_outbound_sockets=127.12.234.1
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:36545
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:35085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:46:42 dist-test-slave-1jjb krb5kdc[25892](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786250802, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 04:46:42.516359 26517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:46:42.516654 26517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:46:42.516826 26517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:46:42.527098 26517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:46:42.527249 26517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:46:42.527429 26517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:46:42.541604 26517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:35085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.12.234.20:47151/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:36545
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:46:42.543192 26517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:46:42.544826 26517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:46:42.564906 26528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:42.569412 26527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:42.569959 26530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:42.570626 26517 server_base.cc:1034] running on GCE node
I20260809 04:46:42.571264 26517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:46:42.572240 26517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:46:42.573453 26517 hybrid_clock.cc:648] HybridClock initialized: now 1786250802573407 us; error 43 us; skew 500 ppm
I20260809 04:46:42.573737 26517 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:46:42 dist-test-slave-1jjb krb5kdc[25892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250802, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:46:42.576754 26517 init.cc:377] Logged in from keytab as kudu/127.12.234.1@KRBTEST.COM (short username kudu)
I20260809 04:46:42.578341 26517 webserver.cc:460] Webserver started at http://127.12.234.1:39443/ using document root <none> and password file <none>
I20260809 04:46:42.579020 26517 fs_manager.cc:359] Metadata directory not provided
I20260809 04:46:42.579141 26517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:46:42.579453 26517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:46:42 dist-test-slave-1jjb krb5kdc[25892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250802, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for HTTP/127.12.234.20@KRBTEST.COM
I20260809 04:46:42.619405 26517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/data/instance:
uuid: "94f80b3b548a4b5ab68470cb45517378"
format_stamp: "Formatted at 2026-08-09 04:46:42 on dist-test-slave-1jjb"
server_key: "4d09dae7f29c632053f0c3e65eb6bfcd"
server_key_iv: "f45c9bbe9dd23dbdf93e88bd096840e2"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:42.620623 26517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance:
uuid: "94f80b3b548a4b5ab68470cb45517378"
format_stamp: "Formatted at 2026-08-09 04:46:42 on dist-test-slave-1jjb"
server_key: "4d09dae7f29c632053f0c3e65eb6bfcd"
server_key_iv: "f45c9bbe9dd23dbdf93e88bd096840e2"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:42.628137 26517 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260809 04:46:42.655361 26537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:42.656944 26517 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.001s sys 0.004s
I20260809 04:46:42.657949 26517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "94f80b3b548a4b5ab68470cb45517378"
format_stamp: "Formatted at 2026-08-09 04:46:42 on dist-test-slave-1jjb"
server_key: "4d09dae7f29c632053f0c3e65eb6bfcd"
server_key_iv: "f45c9bbe9dd23dbdf93e88bd096840e2"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:42.658218 26517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:46:42.686543 26517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:46:42.691761 26517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:46:42.692219 26517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:46:42.693385 26517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:46:42.695118 26517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:46:42.695218 26517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:42.695343 26517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:46:42.695405 26517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:46:42 dist-test-slave-1jjb krb5kdc[25892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250802, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for HTTP/127.12.234.20@KRBTEST.COM
I20260809 04:46:43.052420 26666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:37209 every 8 connection(s)
I20260809 04:46:43.052783 26517 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:37209
I20260809 04:46:43.054486 26517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:46:43.059856 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 26517
I20260809 04:46:43.060045 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance
I20260809 04:46:43.060827 13224 external_mini_cluster.cc:1428] Setting key 6723f0cdd8b6490a79dae9cc749c95e7
Aug 09 04:46:43 dist-test-slave-1jjb krb5kdc[25892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250802, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:46:43.109928 26669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:43.061988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:36545 (local address 127.12.234.1:52997)
0809 04:46:43.076056 (+ 14068us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:43.076136 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:43.078315 (+ 2179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:43.091590 (+ 13275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:43.091601 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:43.092190 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:46:43.093059 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:43.093084 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:43.094650 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:43.094655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:43.096647 (+ 1992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:43.096683 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:43.097900 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:43.097908 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:43.098156 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:43.098951 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:46:43.098981 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:46:43.101525 (+ 2544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:46:43.105315 (+ 3790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:43.105325 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:43.105341 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:43.105897 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:43.106307 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:43.106325 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:43.106328 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:43.106559 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:43.107251 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:46:43.107259 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:46:43.107717 (+ 458us) client_negotiation.cc:769] Sending connection context
0809 04:46:43.108025 (+ 308us) client_negotiation.cc:240] Negotiation successful
0809 04:46:43.108544 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13775,"thread_start_us":16609,"threads_started":1}
I20260809 04:46:43.111109 26668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:43.061515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:52997 (local address 127.12.234.62:36545)
0809 04:46:43.090860 (+ 29345us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:43.090865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:43.090886 (+ 21us) server_negotiation.cc:407] Connection header received
0809 04:46:43.090944 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:43.090948 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:43.091042 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:43.091148 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:43.093268 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:43.094446 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:43.096854 (+ 2408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:43.097461 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:43.097569 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:43.101776 (+ 4207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:43.101782 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:43.101787 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:43.101823 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:43.105157 (+ 3334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:43.106033 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:43.106038 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:43.106042 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:43.106186 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:43.106680 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:43.106685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:43.106687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:43.107002 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:43.107130 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:43.110535 (+ 3405us) server_negotiation.cc:299] Negotiation successful
0809 04:46:43.110731 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29226,"thread_start_us":29298,"threads_started":1}
I20260809 04:46:43.112587 26667 heartbeater.cc:347] Connected to a master server at 127.12.234.62:36545
I20260809 04:46:43.112869 26667 heartbeater.cc:464] Registering TS with master...
I20260809 04:46:43.113564 26667 heartbeater.cc:511] Master 127.12.234.62:36545 requested a full tablet report, sending...
I20260809 04:46:43.115947 26430 ts_manager.cc:194] Registered new tserver with Master: 94f80b3b548a4b5ab68470cb45517378 (127.12.234.1:37209)
Aug 09 04:46:43 dist-test-slave-1jjb krb5kdc[25892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250802, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for HTTP/127.12.234.20@KRBTEST.COM
I20260809 04:46:43.118119 26430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.1@KRBTEST.COM'} at 127.12.234.1:52997
WARNING: no policy specified for kudu/127.12.234.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
W20260809 04:46:43.171712 13224 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.12.234.2: real 0.108s user 0.006s sys 0.016s
I20260809 04:46:43.172569 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.234.20:47151/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:0
--local_ip_for_outbound_sockets=127.12.234.2
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:36545
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:35085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:46:43.564790 26675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:46:43.565062 26675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:46:43.565227 26675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:46:43.572045 26675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:46:43.572182 26675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:46:43.572364 26675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:46:43.580953 26675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:35085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.12.234.20:47151/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:36545
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:46:43.582078 26675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:46:43.583294 26675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:46:43.595765 26689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:43.595944 26690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:43.596278 26692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:43.597018 26675 server_base.cc:1034] running on GCE node
I20260809 04:46:43.597559 26675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:46:43.598407 26675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:46:43.599622 26675 hybrid_clock.cc:648] HybridClock initialized: now 1786250803599582 us; error 56 us; skew 500 ppm
I20260809 04:46:43.599934 26675 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:46:43 dist-test-slave-1jjb krb5kdc[25892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250803, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:46:43.603056 26675 init.cc:377] Logged in from keytab as kudu/127.12.234.2@KRBTEST.COM (short username kudu)
I20260809 04:46:43.604427 26675 webserver.cc:460] Webserver started at http://127.12.234.2:43771/ using document root <none> and password file <none>
I20260809 04:46:43.604966 26675 fs_manager.cc:359] Metadata directory not provided
I20260809 04:46:43.605057 26675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:46:43.605275 26675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:46:43 dist-test-slave-1jjb krb5kdc[25892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250803, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for HTTP/127.12.234.20@KRBTEST.COM
I20260809 04:46:43.619940 26675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/data/instance:
uuid: "47e6dbc35e144f3292038117abd975f4"
format_stamp: "Formatted at 2026-08-09 04:46:43 on dist-test-slave-1jjb"
server_key: "fd1726e94c248c62d741adf0193183b1"
server_key_iv: "c8333ecde12aa64f5e203dc89a80bafc"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:43.620769 26675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance:
uuid: "47e6dbc35e144f3292038117abd975f4"
format_stamp: "Formatted at 2026-08-09 04:46:43 on dist-test-slave-1jjb"
server_key: "fd1726e94c248c62d741adf0193183b1"
server_key_iv: "c8333ecde12aa64f5e203dc89a80bafc"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:43.625584 26675 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:46:43.640489 26701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:43.642023 26675 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 04:46:43.642181 26675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "47e6dbc35e144f3292038117abd975f4"
format_stamp: "Formatted at 2026-08-09 04:46:43 on dist-test-slave-1jjb"
server_key: "fd1726e94c248c62d741adf0193183b1"
server_key_iv: "c8333ecde12aa64f5e203dc89a80bafc"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:43.642421 26675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:46:43.667397 26675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:46:43.672159 26675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:46:43.672477 26675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:46:43.673516 26675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:46:43.675058 26675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:46:43.675179 26675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:43.675294 26675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:46:43.675351 26675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:43.716710 26675 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:32789
I20260809 04:46:43.716717 26828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:32789 every 8 connection(s)
I20260809 04:46:43.718139 26675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:46:43.727767 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 26675
I20260809 04:46:43.727952 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance
I20260809 04:46:43.728515 13224 external_mini_cluster.cc:1428] Setting key d73d0cc3660ea648fd6b87da331ba99b
Aug 09 04:46:43 dist-test-slave-1jjb krb5kdc[25892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250803, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:46:43.740152 26832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:43.720785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:47545 (local address 127.12.234.62:36545)
0809 04:46:43.721525 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:43.721535 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:43.722272 (+ 737us) server_negotiation.cc:407] Connection header received
0809 04:46:43.724034 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:43.724039 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:43.724183 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:43.724286 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:43.725503 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:43.726367 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:43.728160 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:43.728753 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:43.728862 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:43.732947 (+ 4085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:43.732955 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:43.732962 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:43.733001 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:43.736467 (+ 3466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:43.737256 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:43.737264 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:43.737269 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:43.737407 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:43.737856 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:43.737863 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:43.737867 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:43.738201 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:43.738326 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:43.739288 (+ 962us) server_negotiation.cc:299] Negotiation successful
0809 04:46:43.739590 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":519,"threads_started":1}
I20260809 04:46:43.740828 26833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:43.721416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:36545 (local address 127.12.234.2:47545)
0809 04:46:43.722106 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:43.722139 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:43.723792 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:43.724496 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:43.724503 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:43.724846 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:46:43.725348 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:43.725363 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:43.726562 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:43.726566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:43.727957 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:43.727985 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:43.729215 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:43.729221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:43.729440 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:43.730158 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:46:43.730180 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:46:43.732716 (+ 2536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:46:43.736618 (+ 3902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:43.736626 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:43.736638 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:43.737135 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:43.737535 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:43.737539 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:43.737543 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:43.737747 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:43.738353 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:46:43.738360 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:46:43.738752 (+ 392us) client_negotiation.cc:769] Sending connection context
0809 04:46:43.739032 (+ 280us) client_negotiation.cc:240] Negotiation successful
0809 04:46:43.739520 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":294,"threads_started":1}
I20260809 04:46:43.742355 26829 heartbeater.cc:347] Connected to a master server at 127.12.234.62:36545
I20260809 04:46:43.742625 26829 heartbeater.cc:464] Registering TS with master...
I20260809 04:46:43.743181 26829 heartbeater.cc:511] Master 127.12.234.62:36545 requested a full tablet report, sending...
I20260809 04:46:43.744524 26430 ts_manager.cc:194] Registered new tserver with Master: 47e6dbc35e144f3292038117abd975f4 (127.12.234.2:32789)
I20260809 04:46:43.745527 26430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.2@KRBTEST.COM'} at 127.12.234.2:47545
WARNING: no policy specified for kudu/127.12.234.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
I20260809 04:46:43.785450 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.234.20:47151/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:0
--local_ip_for_outbound_sockets=127.12.234.3
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:36545
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:35085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:46:44.121917 26667 heartbeater.cc:503] Master 127.12.234.62:36545 was elected leader, sending a full tablet report...
W20260809 04:46:44.134411 26836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:46:44.134644 26836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:46:44.134764 26836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:46:44.141606 26836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:46:44.141711 26836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:46:44.141872 26836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:46:44.151410 26836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:35085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.12.234.20:47151/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:36545
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:46:44.152489 26836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:46:44.153563 26836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:46:44.165279 26842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:44.165246 26845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:44.165314 26843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:44.165889 26836 server_base.cc:1034] running on GCE node
I20260809 04:46:44.166378 26836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:46:44.167201 26836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:46:44.168418 26836 hybrid_clock.cc:648] HybridClock initialized: now 1786250804168380 us; error 50 us; skew 500 ppm
I20260809 04:46:44.168708 26836 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:46:44 dist-test-slave-1jjb krb5kdc[25892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250804, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:46:44.171638 26836 init.cc:377] Logged in from keytab as kudu/127.12.234.3@KRBTEST.COM (short username kudu)
I20260809 04:46:44.172902 26836 webserver.cc:460] Webserver started at http://127.12.234.3:41471/ using document root <none> and password file <none>
I20260809 04:46:44.173488 26836 fs_manager.cc:359] Metadata directory not provided
I20260809 04:46:44.173621 26836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:46:44.173899 26836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 04:46:44 dist-test-slave-1jjb krb5kdc[25892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250804, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for HTTP/127.12.234.20@KRBTEST.COM
I20260809 04:46:44.190534 26836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/data/instance:
uuid: "9b05f075731b43a88f0c881bf3c91b8a"
format_stamp: "Formatted at 2026-08-09 04:46:44 on dist-test-slave-1jjb"
server_key: "29a1237920486c0fdf40399a41b90cb7"
server_key_iv: "a677844c610c3d8057e447a1a9b3e6ed"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:44.191468 26836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance:
uuid: "9b05f075731b43a88f0c881bf3c91b8a"
format_stamp: "Formatted at 2026-08-09 04:46:44 on dist-test-slave-1jjb"
server_key: "29a1237920486c0fdf40399a41b90cb7"
server_key_iv: "a677844c610c3d8057e447a1a9b3e6ed"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:44.196910 26836 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 04:46:44.212424 26854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:44.213647 26836 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:46:44.213860 26836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "9b05f075731b43a88f0c881bf3c91b8a"
format_stamp: "Formatted at 2026-08-09 04:46:44 on dist-test-slave-1jjb"
server_key: "29a1237920486c0fdf40399a41b90cb7"
server_key_iv: "a677844c610c3d8057e447a1a9b3e6ed"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:44.214056 26836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:46:44.228760 26836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:46:44.231913 26836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:46:44.232175 26836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:46:44.232988 26836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:46:44.234180 26836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:46:44.234244 26836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:44.234400 26836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:46:44.234447 26836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:44.266538 26836 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:33673
I20260809 04:46:44.266577 26973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:33673 every 8 connection(s)
I20260809 04:46:44.267724 26836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:46:44.269508 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 26836
I20260809 04:46:44.269694 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance
I20260809 04:46:44.270288 13224 external_mini_cluster.cc:1428] Setting key 038b09530a624625f56a13b06b93269d
Aug 09 04:46:44 dist-test-slave-1jjb krb5kdc[25892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250804, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:46:44.285590 26977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:44.270199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:53907 (local address 127.12.234.62:36545)
0809 04:46:44.270761 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:44.270765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:44.271707 (+ 942us) server_negotiation.cc:407] Connection header received
0809 04:46:44.273365 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:44.273368 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:44.273435 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:44.273514 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:44.274632 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:44.275315 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:44.276870 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:44.277269 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:44.277357 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:44.280279 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:44.280283 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:44.280286 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:44.280313 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:44.282499 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:44.283103 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:44.283106 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:44.283108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:44.283215 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:44.283627 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:44.283633 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:44.283636 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:44.283984 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:44.284366 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:44.285080 (+ 714us) server_negotiation.cc:299] Negotiation successful
0809 04:46:44.285273 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":338,"threads_started":1}
I20260809 04:46:44.286185 26978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:44.270747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:36545 (local address 127.12.234.3:53907)
0809 04:46:44.271543 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:44.271599 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:44.273157 (+ 1558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:44.273692 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:44.273698 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:44.274035 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:46:44.274511 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:44.274526 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:44.275453 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:44.275456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:44.276727 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:44.276749 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:44.277450 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:44.277455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:44.277645 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:44.278169 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:46:44.278188 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:46:44.280058 (+ 1870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:46:44.282608 (+ 2550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:44.282614 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:44.282623 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:44.282975 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:44.283344 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:44.283346 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:44.283349 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:44.283506 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:44.284291 (+ 785us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:46:44.284298 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:46:44.284625 (+ 327us) client_negotiation.cc:769] Sending connection context
0809 04:46:44.284915 (+ 290us) client_negotiation.cc:240] Negotiation successful
0809 04:46:44.285252 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":334,"threads_started":1}
I20260809 04:46:44.287317 26975 heartbeater.cc:347] Connected to a master server at 127.12.234.62:36545
I20260809 04:46:44.287623 26975 heartbeater.cc:464] Registering TS with master...
I20260809 04:46:44.288285 26975 heartbeater.cc:511] Master 127.12.234.62:36545 requested a full tablet report, sending...
I20260809 04:46:44.289644 26430 ts_manager.cc:194] Registered new tserver with Master: 9b05f075731b43a88f0c881bf3c91b8a (127.12.234.3:33673)
I20260809 04:46:44.290426 26430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.3@KRBTEST.COM'} at 127.12.234.3:53907
I20260809 04:46:44.297228 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:46:44.298997 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 26517
Aug 09 04:46:44 dist-test-slave-1jjb krb5kdc[25892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.1@KRBTEST.COM
I20260809 04:46:44.314371 26981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:44.299993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33464 (local address 127.12.234.1:37209)
0809 04:46:44.300623 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:44.300629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:44.300651 (+ 22us) server_negotiation.cc:407] Connection header received
0809 04:46:44.300701 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:44.300706 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:44.300863 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:44.300989 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:44.302389 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:44.303366 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:44.304758 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:44.305226 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:44.305335 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:44.309018 (+ 3683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:44.309028 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:44.309039 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:44.309090 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:44.311797 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:44.312425 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:44.312431 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:44.312433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:44.312600 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:44.312988 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:44.312992 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:44.312994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:44.313294 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:44.313518 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:44.313780 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 04:46:44.314026 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":342,"threads_started":1}
I20260809 04:46:44.315412 26662 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:46:44.392817 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 26517
I20260809 04:46:44.411725 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 26675
Aug 09 04:46:44 dist-test-slave-1jjb krb5kdc[25892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.2@KRBTEST.COM
I20260809 04:46:44.431178 26983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:44.412669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43434 (local address 127.12.234.2:32789)
0809 04:46:44.413138 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:44.413145 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:44.413169 (+ 24us) server_negotiation.cc:407] Connection header received
0809 04:46:44.413226 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:44.413232 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:44.413430 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:44.413595 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:44.414855 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:44.415765 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:44.417087 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:44.417601 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:44.417736 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:44.426017 (+ 8281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:44.426027 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:44.426040 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:44.426077 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:44.428916 (+ 2839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:44.429606 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:44.429613 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:44.429615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:44.429743 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:44.430052 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:44.430055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:44.430057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:44.430372 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:44.430581 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:44.430746 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 04:46:44.430927 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":234,"threads_started":1}
I20260809 04:46:44.432303 26824 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:46:44.508571 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 26675
I20260809 04:46:44.527872 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 26836
Aug 09 04:46:44 dist-test-slave-1jjb krb5kdc[25892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.3@KRBTEST.COM
I20260809 04:46:44.548594 26986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:44.529073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43112 (local address 127.12.234.3:33673)
0809 04:46:44.529858 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:44.529865 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:44.529894 (+ 29us) server_negotiation.cc:407] Connection header received
0809 04:46:44.529975 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:44.529981 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:44.530243 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:44.532329 (+ 2086us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:44.532634 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:44.533958 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:44.535127 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:44.535876 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:44.536002 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:44.540991 (+ 4989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:44.541009 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:44.541030 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:44.541074 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:44.544964 (+ 3890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:44.545599 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:44.545608 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:44.545612 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:44.545845 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:44.546278 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:44.546283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:44.546287 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:44.546826 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:44.547224 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:44.547860 (+ 636us) server_negotiation.cc:299] Negotiation successful
0809 04:46:44.548152 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":450,"threads_started":1}
I20260809 04:46:44.550221 26969 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:46:44.634248 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 26836
I20260809 04:46:44.652472 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 26393
I20260809 04:46:44.653601 26464 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:46:44.750820 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 26393
I20260809 04:46:44.775072 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:36545
--webserver_interface=127.12.234.62
--webserver_port=36335
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:35085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:36545
--ranger_config_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.234.20:47151/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.234.20:47151/kms with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:46:45.084544 26989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:46:45.084789 26989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:46:45.084872 26989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:46:45.091580 26989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:46:45.091674 26989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:46:45.091728 26989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:46:45.091778 26989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:46:45.091820 26989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:46:45.100958 26989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:35085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.234.20:47151/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:36545
--ranger_config_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:36545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=36335
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:46:45.102319 26989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:46:45.103971 26989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:46:45.115274 26995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:45.115238 26996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:45.115315 26998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:45.116182 26989 server_base.cc:1034] running on GCE node
I20260809 04:46:45.116868 26989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:46:45.118180 26989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:46:45.119385 26989 hybrid_clock.cc:648] HybridClock initialized: now 1786250805119328 us; error 45 us; skew 500 ppm
I20260809 04:46:45.119730 26989 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:46:45 dist-test-slave-1jjb krb5kdc[25892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250805, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:46:45.122792 26989 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:46:45.124305 26989 webserver.cc:460] Webserver started at http://127.12.234.62:36335/ using document root <none> and password file <none>
I20260809 04:46:45.124908 26989 fs_manager.cc:359] Metadata directory not provided
I20260809 04:46:45.125051 26989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 04:46:45 dist-test-slave-1jjb krb5kdc[25892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250805, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for HTTP/127.12.234.20@KRBTEST.COM
I20260809 04:46:45.173074 26989 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260809 04:46:45.175819 27009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:45.177179 26989 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:46:45.177351 26989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "c5d11477f1404dce8742781cea6893c3"
format_stamp: "Formatted at 2026-08-09 04:46:41 on dist-test-slave-1jjb"
server_key: "cc312fcc15494a797764b9c2db1fffeb"
server_key_iv: "c4d331e3a3a50e918c0667238eff5011"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:45.178220 26989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:46:45.198514 26989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:46:45.202042 26989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:46:45.202313 26989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:46:45.220533 26989 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:36545
I20260809 04:46:45.220564 27068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:36545 every 8 connection(s)
I20260809 04:46:45.221813 26989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:46:45.228279 27069 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:46:45.229176 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 26989
I20260809 04:46:45.230295 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.234.20:47151/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:37209
--local_ip_for_outbound_sockets=127.12.234.1
--tserver_master_addrs=127.12.234.62:36545
--webserver_port=39443
--webserver_interface=127.12.234.1
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:35085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.234.20:47151/kms with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:46:45.232931 27069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3: Bootstrap starting.
I20260809 04:46:45.253376 27069 log.cc:824] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3: Log is configured to *not* fsync() on all Append() calls
I20260809 04:46:45.261886 27069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:46:45.262409 27069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3: Bootstrap complete.
I20260809 04:46:45.270651 27069 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d11477f1404dce8742781cea6893c3" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36545 } }
I20260809 04:46:45.271466 27069 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5d11477f1404dce8742781cea6893c3, State: Initialized, Role: FOLLOWER
I20260809 04:46:45.272027 27069 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d11477f1404dce8742781cea6893c3" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36545 } }
I20260809 04:46:45.272308 27069 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:46:45.272449 27069 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:46:45.272624 27069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 04:46:45.276229 27069 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d11477f1404dce8742781cea6893c3" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36545 } }
I20260809 04:46:45.276706 27069 leader_election.cc:302] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5d11477f1404dce8742781cea6893c3; no voters:
I20260809 04:46:45.277345 27069 leader_election.cc:288] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 04:46:45.277473 27073 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 04:46:45.278582 27073 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [term 2 LEADER]: Becoming Leader. State: Replica: c5d11477f1404dce8742781cea6893c3, State: Running, Role: LEADER
I20260809 04:46:45.279137 27073 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d11477f1404dce8742781cea6893c3" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36545 } }
I20260809 04:46:45.279448 27069 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:46:45.281697 27075 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5d11477f1404dce8742781cea6893c3. Latest consensus state: current_term: 2 leader_uuid: "c5d11477f1404dce8742781cea6893c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d11477f1404dce8742781cea6893c3" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36545 } } }
I20260809 04:46:45.282114 27075 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [sys.catalog]: This master's current role is: LEADER
I20260809 04:46:45.282760 27074 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c5d11477f1404dce8742781cea6893c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5d11477f1404dce8742781cea6893c3" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36545 } } }
I20260809 04:46:45.283094 27074 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3 [sys.catalog]: This master's current role is: LEADER
I20260809 04:46:45.291817 27078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:46:45.297117 27078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:46:45.298454 27078 catalog_manager.cc:1259] Loaded cluster ID: 7d8614f8935f43f3b55d0d99c4284cc3
I20260809 04:46:45.299194 27078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:46:45.304397 27078 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:46:45.306159 27078 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3: Loaded TSK: 0
I20260809 04:46:45.307864 27078 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:46:45.311869 27069 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260809 04:46:45.660712 27071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:46:45.661062 27071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:46:45.661237 27071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:46:45.672092 27071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:46:45.672212 27071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:46:45.672379 27071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:46:45.687480 27071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:35085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.234.20:47151/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:37209
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=39443
--tserver_master_addrs=127.12.234.62:36545
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:46:45.689131 27071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:46:45.690879 27071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:46:45.743125 27112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:45.745332 27110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:45.750605 27115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:45.752332 27071 server_base.cc:1034] running on GCE node
I20260809 04:46:45.753065 27071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:46:45.754315 27071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:46:45.755558 27071 hybrid_clock.cc:648] HybridClock initialized: now 1786250805755506 us; error 38 us; skew 500 ppm
I20260809 04:46:45.755965 27071 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:46:45 dist-test-slave-1jjb krb5kdc[25892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250805, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:46:45.760090 27071 init.cc:377] Logged in from keytab as kudu/127.12.234.1@KRBTEST.COM (short username kudu)
I20260809 04:46:45.781342 27071 webserver.cc:460] Webserver started at http://127.12.234.1:39443/ using document root <none> and password file <none>
I20260809 04:46:45.782347 27071 fs_manager.cc:359] Metadata directory not provided
I20260809 04:46:45.782510 27071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 04:46:45 dist-test-slave-1jjb krb5kdc[25892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250805, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for HTTP/127.12.234.20@KRBTEST.COM
I20260809 04:46:45.861542 27071 fs_manager.cc:709] Time spent opening directory manager: real 0.019s user 0.007s sys 0.005s
I20260809 04:46:45.872646 27124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:45.875690 27071 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.005s sys 0.000s
I20260809 04:46:45.875852 27071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "94f80b3b548a4b5ab68470cb45517378"
format_stamp: "Formatted at 2026-08-09 04:46:42 on dist-test-slave-1jjb"
server_key: "4d09dae7f29c632053f0c3e65eb6bfcd"
server_key_iv: "f45c9bbe9dd23dbdf93e88bd096840e2"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:45.877177 27071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:46:45.910636 27071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:46:45.914585 27071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:46:45.914856 27071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:46:45.915827 27071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:46:45.917260 27071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:46:45.917368 27071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:45.917485 27071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:46:45.917589 27071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:45.971699 27071 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:37209
I20260809 04:46:45.971750 27247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:37209 every 8 connection(s)
I20260809 04:46:45.973390 27071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:46:45.979238 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 27071
I20260809 04:46:45.980356 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.234.20:47151/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:32789
--local_ip_for_outbound_sockets=127.12.234.2
--tserver_master_addrs=127.12.234.62:36545
--webserver_port=43771
--webserver_interface=127.12.234.2
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:35085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.234.20:47151/kms with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:46:46 dist-test-slave-1jjb krb5kdc[25892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250805, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:46:46.030575 27250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:45.979321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:36545 (local address 127.12.234.1:33799)
0809 04:46:46.005462 (+ 26141us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:46.005524 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:46.007955 (+ 2431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:46.009794 (+ 1839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:46.009807 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:46.010517 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:46:46.011271 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:46.011297 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:46.013597 (+ 2300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:46.013604 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:46.015733 (+ 2129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:46.015763 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:46.017162 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:46.017172 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:46.017475 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:46.018272 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:46:46.018303 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:46:46.021374 (+ 3071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:46:46.025650 (+ 4276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:46.025658 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:46.025671 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:46.026212 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:46.026738 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:46.026742 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:46.026745 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:46.026963 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:46.027983 (+ 1020us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:46:46.027990 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:46:46.028358 (+ 368us) client_negotiation.cc:769] Sending connection context
0809 04:46:46.028643 (+ 285us) client_negotiation.cc:240] Negotiation successful
0809 04:46:46.029217 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25813,"thread_start_us":25568,"threads_started":1}
I20260809 04:46:46.032932 27251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:45.980442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:33799 (local address 127.12.234.62:36545)
0809 04:46:46.004813 (+ 24371us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:46.004837 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:46.006015 (+ 1178us) server_negotiation.cc:407] Connection header received
0809 04:46:46.008698 (+ 2683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:46.008719 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:46.009223 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:46.009636 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:46.011519 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:46.013420 (+ 1901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:46.015917 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:46.016922 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:46.017118 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:46.021682 (+ 4564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:46.021694 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:46.021712 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:46.021757 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:46.025491 (+ 3734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:46.026400 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:46.026409 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:46.026417 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:46.026597 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:46.027099 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:46.027104 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:46.027107 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:46.027707 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:46.027892 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:46.031377 (+ 3485us) server_negotiation.cc:299] Negotiation successful
0809 04:46:46.031762 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24093,"thread_start_us":23864,"threads_started":1}
I20260809 04:46:46.035696 27248 heartbeater.cc:347] Connected to a master server at 127.12.234.62:36545
I20260809 04:46:46.036012 27248 heartbeater.cc:464] Registering TS with master...
I20260809 04:46:46.036738 27248 heartbeater.cc:511] Master 127.12.234.62:36545 requested a full tablet report, sending...
W20260809 04:46:46.039407 27248 heartbeater.cc:644] Failed to heartbeat to 127.12.234.62:36545 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 04:46:46.039649 27248 heartbeater.cc:464] Registering TS with master...
I20260809 04:46:46.039954 27248 heartbeater.cc:511] Master 127.12.234.62:36545 requested a full tablet report, sending...
I20260809 04:46:46.041150 27248 heartbeater.cc:464] Registering TS with master...
I20260809 04:46:46.041388 27248 heartbeater.cc:511] Master 127.12.234.62:36545 requested a full tablet report, sending...
W20260809 04:46:46.042326 27248 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 04:46:46.415320 27252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:46:46.415663 27252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:46:46.415877 27252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:46:46.427078 27252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:46:46.427268 27252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:46:46.427475 27252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:46:46.439272 27252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:35085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.234.20:47151/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:32789
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=43771
--tserver_master_addrs=127.12.234.62:36545
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:46:46.440677 27252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:46:46.441897 27252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:46:46.458851 27270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:46.460453 27269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:46.462159 27272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:46.462579 27252 server_base.cc:1034] running on GCE node
I20260809 04:46:46.463212 27252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:46:46.464221 27252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:46:46.465449 27252 hybrid_clock.cc:648] HybridClock initialized: now 1786250806465402 us; error 42 us; skew 500 ppm
I20260809 04:46:46.465755 27252 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:46:46 dist-test-slave-1jjb krb5kdc[25892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250806, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:46:46.472827 27252 init.cc:377] Logged in from keytab as kudu/127.12.234.2@KRBTEST.COM (short username kudu)
I20260809 04:46:46.474534 27252 webserver.cc:460] Webserver started at http://127.12.234.2:43771/ using document root <none> and password file <none>
I20260809 04:46:46.475169 27252 fs_manager.cc:359] Metadata directory not provided
I20260809 04:46:46.475315 27252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 04:46:46 dist-test-slave-1jjb krb5kdc[25892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250806, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for HTTP/127.12.234.20@KRBTEST.COM
Aug 09 04:46:46 dist-test-slave-1jjb krb5kdc[25892](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786250806, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:46:46.500694 27252 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.002s sys 0.005s
I20260809 04:46:46.504117 27280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:46.505810 27252 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:46:46.505972 27252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "47e6dbc35e144f3292038117abd975f4"
format_stamp: "Formatted at 2026-08-09 04:46:43 on dist-test-slave-1jjb"
server_key: "fd1726e94c248c62d741adf0193183b1"
server_key_iv: "c8333ecde12aa64f5e203dc89a80bafc"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:46.507282 27252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:46:46.554420 27252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:46:46.558668 27252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:46:46.558976 27252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:46:46.560055 27252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:46:46.561472 27252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:46:46.561553 27252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:46.561691 27252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:46:46.561759 27252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:46.603653 27252 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:32789
I20260809 04:46:46.603716 27407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:32789 every 8 connection(s)
I20260809 04:46:46.605305 27252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:46:46.615402 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 27252
I20260809 04:46:46.616439 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.234.20:47151/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:33673
--local_ip_for_outbound_sockets=127.12.234.3
--tserver_master_addrs=127.12.234.62:36545
--webserver_port=41471
--webserver_interface=127.12.234.3
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:35085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.234.20:47151/kms with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:46:46 dist-test-slave-1jjb krb5kdc[25892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250806, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:46:46.647712 27411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:46.609092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:36545 (local address 127.12.234.2:34001)
0809 04:46:46.620888 (+ 11796us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:46.620947 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:46.623459 (+ 2512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:46.628316 (+ 4857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:46.628329 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:46.628966 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:46:46.629545 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:46.629567 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:46.631134 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:46.631140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:46.633224 (+ 2084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:46.633243 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:46.634433 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:46.634442 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:46.634778 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:46.635775 (+ 997us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:46:46.635806 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:46:46.638623 (+ 2817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:46:46.642873 (+ 4250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:46.642884 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:46.642900 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:46.643518 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:46.643954 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:46.643961 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:46.643968 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:46.644216 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:46.644945 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:46:46.644955 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:46:46.645371 (+ 416us) client_negotiation.cc:769] Sending connection context
0809 04:46:46.645661 (+ 290us) client_negotiation.cc:240] Negotiation successful
0809 04:46:46.646186 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11491,"thread_start_us":17227,"threads_started":1}
I20260809 04:46:46.649782 27410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:46.608420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:34001 (local address 127.12.234.62:36545)
0809 04:46:46.627430 (+ 19010us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:46.627438 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:46.627462 (+ 24us) server_negotiation.cc:407] Connection header received
0809 04:46:46.627533 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:46.627541 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:46.627650 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:46.627774 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:46.629739 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:46.630933 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:46.633462 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:46.634191 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:46.635176 (+ 985us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:46.638880 (+ 3704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:46.638888 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:46.638895 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:46.638957 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:46.642728 (+ 3771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:46.643674 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:46.643681 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:46.643686 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:46.643826 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:46.644374 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:46.644381 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:46.644385 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:46.644786 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:46.648800 (+ 4014us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:46.648995 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 04:46:46.649279 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18851,"thread_start_us":19478,"threads_started":1}
I20260809 04:46:46.651270 27408 heartbeater.cc:347] Connected to a master server at 127.12.234.62:36545
I20260809 04:46:46.651592 27408 heartbeater.cc:464] Registering TS with master...
I20260809 04:46:46.652472 27408 heartbeater.cc:511] Master 127.12.234.62:36545 requested a full tablet report, sending...
I20260809 04:46:46.654479 27032 ts_manager.cc:194] Registered new tserver with Master: 47e6dbc35e144f3292038117abd975f4 (127.12.234.2:32789)
I20260809 04:46:46.656464 27032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.2@KRBTEST.COM'} at 127.12.234.2:34001
Aug 09 04:46:46 dist-test-slave-1jjb krb5kdc[25892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250806, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for HTTP/127.12.234.20@KRBTEST.COM
W20260809 04:46:47.021483 27412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:46:47.021756 27412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:46:47.021926 27412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:46:47.028753 27412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:46:47.028878 27412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:46:47.029094 27412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:46:47.038157 27412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:35085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.234.20:47151/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:33673
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=41471
--tserver_master_addrs=127.12.234.62:36545
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:46:47.039314 27412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:46:47.040879 27412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:46:47.042495 27248 heartbeater.cc:464] Registering TS with master...
I20260809 04:46:47.042907 27248 heartbeater.cc:511] Master 127.12.234.62:36545 requested a full tablet report, sending...
I20260809 04:46:47.044106 27032 ts_manager.cc:194] Registered new tserver with Master: 94f80b3b548a4b5ab68470cb45517378 (127.12.234.1:37209)
I20260809 04:46:47.045279 27032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.1@KRBTEST.COM'} at 127.12.234.1:33799
W20260809 04:46:47.056034 27418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:47.056035 27419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:47.063581 27421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:47.064128 27412 server_base.cc:1034] running on GCE node
I20260809 04:46:47.064757 27412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:46:47.065732 27412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:46:47.066987 27412 hybrid_clock.cc:648] HybridClock initialized: now 1786250807066933 us; error 31 us; skew 500 ppm
I20260809 04:46:47.067312 27412 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:46:47 dist-test-slave-1jjb krb5kdc[25892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250807, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:46:47.071311 27412 init.cc:377] Logged in from keytab as kudu/127.12.234.3@KRBTEST.COM (short username kudu)
I20260809 04:46:47.073143 27412 webserver.cc:460] Webserver started at http://127.12.234.3:41471/ using document root <none> and password file <none>
I20260809 04:46:47.073959 27412 fs_manager.cc:359] Metadata directory not provided
I20260809 04:46:47.074086 27412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 04:46:47 dist-test-slave-1jjb krb5kdc[25892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250807, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for HTTP/127.12.234.20@KRBTEST.COM
I20260809 04:46:47.112186 27412 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.002s sys 0.005s
I20260809 04:46:47.115564 27430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:47.117676 27412 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 04:46:47.117877 27412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "9b05f075731b43a88f0c881bf3c91b8a"
format_stamp: "Formatted at 2026-08-09 04:46:44 on dist-test-slave-1jjb"
server_key: "29a1237920486c0fdf40399a41b90cb7"
server_key_iv: "a677844c610c3d8057e447a1a9b3e6ed"
server_key_version: "kuduclusterkey@0"
I20260809 04:46:47.119223 27412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:46:47.152064 27412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:46:47.156399 27412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:46:47.156694 27412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:46:47.157814 27412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:46:47.159533 27412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:46:47.159684 27412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 04:46:47.159818 27412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:46:47.159940 27412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 04:46:47 dist-test-slave-1jjb krb5kdc[25892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250806, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for HTTP/127.12.234.20@KRBTEST.COM
I20260809 04:46:47.321316 27412 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:33673
I20260809 04:46:47.321382 27557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:33673 every 8 connection(s)
I20260809 04:46:47.323122 27412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:46:47.331758 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 27412
I20260809 04:46:47.379848 27561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:47.333005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53718 (local address 127.12.234.62:36545)
0809 04:46:47.343335 (+ 10330us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:47.343343 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:47.356401 (+ 13058us) server_negotiation.cc:407] Connection header received
0809 04:46:47.356486 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:47.356490 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:47.356585 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:47.356705 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:47.358955 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.360103 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.362541 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.363150 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.363270 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:47.367297 (+ 4027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:47.367303 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:47.367308 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:47.367339 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:47.373135 (+ 5796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:47.374379 (+ 1244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:47.374384 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:47.374387 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:47.374548 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:47.378181 (+ 3633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:47.378188 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:47.378191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:47.378535 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:47.378848 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:47.379150 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 04:46:47.379414 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10175,"thread_start_us":11060,"threads_started":1}
Aug 09 04:46:47 dist-test-slave-1jjb krb5kdc[25892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250807, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:46:47.413754 27564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:47.347414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:53373 (local address 127.12.234.62:36545)
0809 04:46:47.367205 (+ 19791us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:47.367210 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:47.374323 (+ 7113us) server_negotiation.cc:407] Connection header received
0809 04:46:47.378480 (+ 4157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:47.378485 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:47.378580 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:47.378669 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:47.381341 (+ 2672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.383023 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.390635 (+ 7612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.391225 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.391316 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:47.397357 (+ 6041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:47.397364 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:47.397369 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:47.397404 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:47.402009 (+ 4605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:47.405542 (+ 3533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:47.405546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:47.405549 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:47.405683 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:47.411778 (+ 6095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:47.411782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:47.411786 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:47.412156 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:47.412277 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:47.413138 (+ 861us) server_negotiation.cc:299] Negotiation successful
0809 04:46:47.413399 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19679,"thread_start_us":19550,"threads_started":1}
I20260809 04:46:47.415974 27563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:47.346772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:36545 (local address 127.12.234.3:53373)
0809 04:46:47.373800 (+ 27028us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:47.373858 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:47.378082 (+ 4224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:47.379161 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:47.379173 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:47.379768 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:46:47.380621 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:47.380649 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.383259 (+ 2610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.383266 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:47.390477 (+ 7211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:47.390492 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.392062 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.392070 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:47.392916 (+ 846us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:47.393792 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:46:47.393854 (+ 62us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:46:47.397067 (+ 3213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:46:47.402187 (+ 5120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:47.402196 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:47.402212 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:47.404176 (+ 1964us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:47.411408 (+ 7232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:47.411428 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:47.411432 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:47.411654 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:47.412358 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:46:47.412366 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:46:47.412756 (+ 390us) client_negotiation.cc:769] Sending connection context
0809 04:46:47.414214 (+ 1458us) client_negotiation.cc:240] Negotiation successful
0809 04:46:47.414718 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26722,"thread_start_us":26432,"threads_started":1}
I20260809 04:46:47.419027 27558 heartbeater.cc:347] Connected to a master server at 127.12.234.62:36545
I20260809 04:46:47.419332 27558 heartbeater.cc:464] Registering TS with master...
I20260809 04:46:47.420244 27558 heartbeater.cc:511] Master 127.12.234.62:36545 requested a full tablet report, sending...
I20260809 04:46:47.421844 27032 ts_manager.cc:194] Registered new tserver with Master: 9b05f075731b43a88f0c881bf3c91b8a (127.12.234.3:33673)
I20260809 04:46:47.422902 27032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.3@KRBTEST.COM'} at 127.12.234.3:53373
I20260809 04:46:47.430657 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:46:47.504160 27564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:47.466928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53722 (local address 127.12.234.62:36545)
0809 04:46:47.467111 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:47.467118 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:47.486268 (+ 19150us) server_negotiation.cc:407] Connection header received
0809 04:46:47.486832 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:47.486838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:47.486931 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:47.487008 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:47.489264 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.490375 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.492360 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.493065 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.493154 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:47.496888 (+ 3734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:47.496898 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:47.496905 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:47.496947 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:47.500562 (+ 3615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:47.501777 (+ 1215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:47.501783 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:47.501787 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:47.501932 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:47.502384 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:47.502389 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:47.502393 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:47.502744 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:47.502969 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:47.503579 (+ 610us) server_negotiation.cc:299] Negotiation successful
0809 04:46:47.503798 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:46:47.510983 27032 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53722:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:46:47.522049 27032 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:46:47.618042 27581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:47.551932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:37209 (local address 127.0.0.1:33470)
0809 04:46:47.594528 (+ 42596us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:47.594576 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:47.594796 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:47.598471 (+ 3675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:47.598479 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:47.598525 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:46:47.598997 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:47.599015 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.603256 (+ 4241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.603262 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:47.614734 (+ 11472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:47.614754 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.617086 (+ 2332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.617093 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:47.617300 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:47.617362 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 04:46:47.617487 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 04:46:47.617666 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42298,"threads_started":1}
I20260809 04:46:47.619139 27578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:47.552032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33470 (local address 127.12.234.1:37209)
0809 04:46:47.595400 (+ 43368us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:47.595408 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:47.595439 (+ 31us) server_negotiation.cc:407] Connection header received
0809 04:46:47.595543 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:47.595550 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:47.595811 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:47.596027 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:46:47.599546 (+ 3519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.601194 (+ 1648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.614993 (+ 13799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.616522 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.616671 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:47.616923 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:47.618620 (+ 1697us) server_negotiation.cc:299] Negotiation successful
0809 04:46:47.618792 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43182,"thread_start_us":55193,"threads_started":1}
I20260809 04:46:47.619683 27580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:47.552886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43114 (local address 127.12.234.3:33673)
0809 04:46:47.589958 (+ 37072us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:47.589969 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:47.602545 (+ 12576us) server_negotiation.cc:407] Connection header received
0809 04:46:47.606254 (+ 3709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:47.606261 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:47.606494 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:47.606653 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:46:47.607768 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.609716 (+ 1948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.615239 (+ 5523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.616722 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.616836 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:47.618087 (+ 1251us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:47.618976 (+ 889us) server_negotiation.cc:299] Negotiation successful
0809 04:46:47.619155 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36853,"thread_start_us":38826,"threads_started":1}
I20260809 04:46:47.620587 27579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:47.552647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:33673 (local address 127.0.0.1:43114)
0809 04:46:47.602433 (+ 49786us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:47.602452 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:47.605470 (+ 3018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:47.607059 (+ 1589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:47.607063 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:47.607090 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:46:47.607482 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:47.607496 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.610694 (+ 3198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.610700 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:47.612769 (+ 2069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:47.612785 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.618560 (+ 5775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.618566 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:47.618742 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:47.618774 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 04:46:47.620106 (+ 1332us) client_negotiation.cc:240] Negotiation successful
0809 04:46:47.620261 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49661,"thread_start_us":49920,"threads_started":1}
I20260809 04:46:47.625190 27484 tablet_service.cc:1467] Processing CreateTablet for tablet d7f2ed9b8d2b4d328d87635437e08233 (DEFAULT_TABLE table=test-table [id=e6f0670652374eb884cdbbc044f6a16e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:46:47.625816 27177 tablet_service.cc:1467] Processing CreateTablet for tablet d7f2ed9b8d2b4d328d87635437e08233 (DEFAULT_TABLE table=test-table [id=e6f0670652374eb884cdbbc044f6a16e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:46:47.626799 27484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7f2ed9b8d2b4d328d87635437e08233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:46:47.627468 27177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7f2ed9b8d2b4d328d87635437e08233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:46:47.630184 27581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:47.553037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:32789 (local address 127.0.0.1:43436)
0809 04:46:47.618289 (+ 65252us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:47.618310 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:47.618431 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:47.620270 (+ 1839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:47.620274 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:47.620297 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:46:47.620447 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:47.620457 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.622279 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.622284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:47.626293 (+ 4009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:47.626309 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.628440 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.628446 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:47.628628 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:47.629699 (+ 1071us) client_negotiation.cc:769] Sending connection context
0809 04:46:47.629778 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 04:46:47.629923 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65124,"thread_start_us":41053,"threads_started":1}
I20260809 04:46:47.631069 27582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:47.553732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43436 (local address 127.12.234.2:32789)
0809 04:46:47.605463 (+ 51731us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:47.605470 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:47.618400 (+ 12930us) server_negotiation.cc:407] Connection header received
0809 04:46:47.619451 (+ 1051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:47.619459 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:47.619690 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:47.619926 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:46:47.620652 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.622107 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.626486 (+ 4379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.627944 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.628062 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:47.628296 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:47.630453 (+ 2157us) server_negotiation.cc:299] Negotiation successful
0809 04:46:47.630643 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51567,"thread_start_us":51447,"threads_started":1}
I20260809 04:46:47.635222 27336 tablet_service.cc:1467] Processing CreateTablet for tablet d7f2ed9b8d2b4d328d87635437e08233 (DEFAULT_TABLE table=test-table [id=e6f0670652374eb884cdbbc044f6a16e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:46:47.636531 27336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7f2ed9b8d2b4d328d87635437e08233. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:46:47.653039 27586 tablet_bootstrap.cc:492] T d7f2ed9b8d2b4d328d87635437e08233 P 9b05f075731b43a88f0c881bf3c91b8a: Bootstrap starting.
I20260809 04:46:47.654903 27589 tablet_bootstrap.cc:492] T d7f2ed9b8d2b4d328d87635437e08233 P 47e6dbc35e144f3292038117abd975f4: Bootstrap starting.
I20260809 04:46:47.656749 27586 tablet_bootstrap.cc:654] T d7f2ed9b8d2b4d328d87635437e08233 P 9b05f075731b43a88f0c881bf3c91b8a: Neither blocks nor log segments found. Creating new log.
I20260809 04:46:47.658102 27586 log.cc:824] T d7f2ed9b8d2b4d328d87635437e08233 P 9b05f075731b43a88f0c881bf3c91b8a: Log is configured to *not* fsync() on all Append() calls
I20260809 04:46:47.658175 27589 tablet_bootstrap.cc:654] T d7f2ed9b8d2b4d328d87635437e08233 P 47e6dbc35e144f3292038117abd975f4: Neither blocks nor log segments found. Creating new log.
I20260809 04:46:47.659552 27589 log.cc:824] T d7f2ed9b8d2b4d328d87635437e08233 P 47e6dbc35e144f3292038117abd975f4: Log is configured to *not* fsync() on all Append() calls
I20260809 04:46:47.660611 27408 heartbeater.cc:503] Master 127.12.234.62:36545 was elected leader, sending a full tablet report...
I20260809 04:46:47.663335 27587 tablet_bootstrap.cc:492] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378: Bootstrap starting.
I20260809 04:46:47.666441 27587 tablet_bootstrap.cc:654] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378: Neither blocks nor log segments found. Creating new log.
I20260809 04:46:47.667562 27587 log.cc:824] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378: Log is configured to *not* fsync() on all Append() calls
I20260809 04:46:47.668650 27586 tablet_bootstrap.cc:492] T d7f2ed9b8d2b4d328d87635437e08233 P 9b05f075731b43a88f0c881bf3c91b8a: No bootstrap required, opened a new log
I20260809 04:46:47.669018 27586 ts_tablet_manager.cc:1397] T d7f2ed9b8d2b4d328d87635437e08233 P 9b05f075731b43a88f0c881bf3c91b8a: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.006s
I20260809 04:46:47.670614 27589 tablet_bootstrap.cc:492] T d7f2ed9b8d2b4d328d87635437e08233 P 47e6dbc35e144f3292038117abd975f4: No bootstrap required, opened a new log
I20260809 04:46:47.670862 27589 ts_tablet_manager.cc:1397] T d7f2ed9b8d2b4d328d87635437e08233 P 47e6dbc35e144f3292038117abd975f4: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.006s
I20260809 04:46:47.679275 27586 raft_consensus.cc:348] T d7f2ed9b8d2b4d328d87635437e08233 P 9b05f075731b43a88f0c881bf3c91b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f80b3b548a4b5ab68470cb45517378" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 37209 } } peers { permanent_uuid: "9b05f075731b43a88f0c881bf3c91b8a" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33673 } } peers { permanent_uuid: "47e6dbc35e144f3292038117abd975f4" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 32789 } }
I20260809 04:46:47.679800 27586 raft_consensus.cc:374] T d7f2ed9b8d2b4d328d87635437e08233 P 9b05f075731b43a88f0c881bf3c91b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:46:47.679953 27586 raft_consensus.cc:729] T d7f2ed9b8d2b4d328d87635437e08233 P 9b05f075731b43a88f0c881bf3c91b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b05f075731b43a88f0c881bf3c91b8a, State: Initialized, Role: FOLLOWER
I20260809 04:46:47.680833 27586 consensus_queue.cc:260] T d7f2ed9b8d2b4d328d87635437e08233 P 9b05f075731b43a88f0c881bf3c91b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f80b3b548a4b5ab68470cb45517378" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 37209 } } peers { permanent_uuid: "9b05f075731b43a88f0c881bf3c91b8a" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33673 } } peers { permanent_uuid: "47e6dbc35e144f3292038117abd975f4" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 32789 } }
I20260809 04:46:47.681336 27589 raft_consensus.cc:348] T d7f2ed9b8d2b4d328d87635437e08233 P 47e6dbc35e144f3292038117abd975f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f80b3b548a4b5ab68470cb45517378" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 37209 } } peers { permanent_uuid: "9b05f075731b43a88f0c881bf3c91b8a" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33673 } } peers { permanent_uuid: "47e6dbc35e144f3292038117abd975f4" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 32789 } }
I20260809 04:46:47.681763 27589 raft_consensus.cc:374] T d7f2ed9b8d2b4d328d87635437e08233 P 47e6dbc35e144f3292038117abd975f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:46:47.681897 27589 raft_consensus.cc:729] T d7f2ed9b8d2b4d328d87635437e08233 P 47e6dbc35e144f3292038117abd975f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47e6dbc35e144f3292038117abd975f4, State: Initialized, Role: FOLLOWER
I20260809 04:46:47.682605 27589 consensus_queue.cc:260] T d7f2ed9b8d2b4d328d87635437e08233 P 47e6dbc35e144f3292038117abd975f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f80b3b548a4b5ab68470cb45517378" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 37209 } } peers { permanent_uuid: "9b05f075731b43a88f0c881bf3c91b8a" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33673 } } peers { permanent_uuid: "47e6dbc35e144f3292038117abd975f4" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 32789 } }
I20260809 04:46:47.684641 27587 tablet_bootstrap.cc:492] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378: No bootstrap required, opened a new log
I20260809 04:46:47.684984 27587 ts_tablet_manager.cc:1397] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378: Time spent bootstrapping tablet: real 0.022s user 0.007s sys 0.012s
I20260809 04:46:47.689093 27586 ts_tablet_manager.cc:1428] T d7f2ed9b8d2b4d328d87635437e08233 P 9b05f075731b43a88f0c881bf3c91b8a: Time spent starting tablet: real 0.020s user 0.013s sys 0.005s
I20260809 04:46:47.689496 27558 heartbeater.cc:503] Master 127.12.234.62:36545 was elected leader, sending a full tablet report...
I20260809 04:46:47.690433 27589 ts_tablet_manager.cc:1428] T d7f2ed9b8d2b4d328d87635437e08233 P 47e6dbc35e144f3292038117abd975f4: Time spent starting tablet: real 0.019s user 0.014s sys 0.005s
I20260809 04:46:47.695019 27587 raft_consensus.cc:348] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f80b3b548a4b5ab68470cb45517378" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 37209 } } peers { permanent_uuid: "9b05f075731b43a88f0c881bf3c91b8a" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33673 } } peers { permanent_uuid: "47e6dbc35e144f3292038117abd975f4" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 32789 } }
I20260809 04:46:47.695343 27587 raft_consensus.cc:374] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:46:47.695446 27587 raft_consensus.cc:729] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94f80b3b548a4b5ab68470cb45517378, State: Initialized, Role: FOLLOWER
I20260809 04:46:47.695906 27587 consensus_queue.cc:260] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f80b3b548a4b5ab68470cb45517378" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 37209 } } peers { permanent_uuid: "9b05f075731b43a88f0c881bf3c91b8a" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33673 } } peers { permanent_uuid: "47e6dbc35e144f3292038117abd975f4" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 32789 } }
I20260809 04:46:47.696861 27248 heartbeater.cc:503] Master 127.12.234.62:36545 was elected leader, sending a full tablet report...
I20260809 04:46:47.697232 27587 ts_tablet_manager.cc:1428] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
W20260809 04:46:47.734272 27249 tablet.cc:2369] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:46:47.857262 27562 tablet.cc:2369] T d7f2ed9b8d2b4d328d87635437e08233 P 9b05f075731b43a88f0c881bf3c91b8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:46:47.863662 27409 tablet.cc:2369] T d7f2ed9b8d2b4d328d87635437e08233 P 47e6dbc35e144f3292038117abd975f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:46:47.911912 27600 raft_consensus.cc:482] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:46:47.912251 27600 raft_consensus.cc:504] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f80b3b548a4b5ab68470cb45517378" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 37209 } } peers { permanent_uuid: "9b05f075731b43a88f0c881bf3c91b8a" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33673 } } peers { permanent_uuid: "47e6dbc35e144f3292038117abd975f4" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 32789 } }
I20260809 04:46:47.913985 27600 leader_election.cc:288] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b05f075731b43a88f0c881bf3c91b8a (127.12.234.3:33673), 47e6dbc35e144f3292038117abd975f4 (127.12.234.2:32789)
I20260809 04:46:47.920449 27582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:47.914464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:41791 (local address 127.12.234.2:32789)
0809 04:46:47.914729 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:47.914737 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:47.915423 (+ 686us) server_negotiation.cc:407] Connection header received
0809 04:46:47.915625 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:47.915634 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:47.915690 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:47.915779 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:46:47.916361 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.917022 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.918725 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.919485 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.919676 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:47.919754 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:47.920027 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 04:46:47.920189 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 04:46:47.920576 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:47.914227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:32789 (local address 127.12.234.1:41791)
0809 04:46:47.915316 (+ 1089us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:47.915331 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:47.915490 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:47.915799 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:47.915803 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:47.915825 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:46:47.916221 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:47.916230 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.917149 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.917155 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:47.918585 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:47.918596 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.919690 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.919696 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:47.919857 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:47.919880 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 04:46:47.920000 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 04:46:47.920151 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":955,"spinlock_wait_cycles":31104,"thread_start_us":731,"threads_started":1}
I20260809 04:46:47.920775 27580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:47.914387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:50785 (local address 127.12.234.3:33673)
0809 04:46:47.914642 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:47.914651 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:47.915501 (+ 850us) server_negotiation.cc:407] Connection header received
0809 04:46:47.915687 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:47.915703 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:47.915758 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:47.915846 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:46:47.916480 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.917123 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.919041 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.919817 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.919989 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:47.920070 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:47.920431 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 04:46:47.920547 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 04:46:47.920825 27603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:47.914362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:33673 (local address 127.12.234.1:50785)
0809 04:46:47.915405 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:47.915422 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:47.915550 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:47.915866 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:47.915869 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:47.915895 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:46:47.916331 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:47.916340 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.917253 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.917257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:47.918884 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:47.918895 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.920044 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.920050 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:47.920211 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:47.920233 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 04:46:47.920365 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 04:46:47.920481 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"thread_start_us":705,"threads_started":1}
I20260809 04:46:47.921589 27361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7f2ed9b8d2b4d328d87635437e08233" candidate_uuid: "94f80b3b548a4b5ab68470cb45517378" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47e6dbc35e144f3292038117abd975f4" is_pre_election: true
I20260809 04:46:47.921674 27508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7f2ed9b8d2b4d328d87635437e08233" candidate_uuid: "94f80b3b548a4b5ab68470cb45517378" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b05f075731b43a88f0c881bf3c91b8a" is_pre_election: true
I20260809 04:46:47.922093 27361 raft_consensus.cc:2454] T d7f2ed9b8d2b4d328d87635437e08233 P 47e6dbc35e144f3292038117abd975f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94f80b3b548a4b5ab68470cb45517378 in term 0.
I20260809 04:46:47.922171 27508 raft_consensus.cc:2454] T d7f2ed9b8d2b4d328d87635437e08233 P 9b05f075731b43a88f0c881bf3c91b8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94f80b3b548a4b5ab68470cb45517378 in term 0.
I20260809 04:46:47.922771 27127 leader_election.cc:302] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 47e6dbc35e144f3292038117abd975f4, 94f80b3b548a4b5ab68470cb45517378, 9b05f075731b43a88f0c881bf3c91b8a; no voters:
I20260809 04:46:47.923166 27600 raft_consensus.cc:2781] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:46:47.923312 27600 raft_consensus.cc:482] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:46:47.923415 27600 raft_consensus.cc:3037] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:46:47.925634 27600 raft_consensus.cc:504] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f80b3b548a4b5ab68470cb45517378" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 37209 } } peers { permanent_uuid: "9b05f075731b43a88f0c881bf3c91b8a" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33673 } } peers { permanent_uuid: "47e6dbc35e144f3292038117abd975f4" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 32789 } }
I20260809 04:46:47.926115 27600 leader_election.cc:288] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378 [CANDIDATE]: Term 1 election: Requested vote from peers 9b05f075731b43a88f0c881bf3c91b8a (127.12.234.3:33673), 47e6dbc35e144f3292038117abd975f4 (127.12.234.2:32789)
I20260809 04:46:47.926543 27508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7f2ed9b8d2b4d328d87635437e08233" candidate_uuid: "94f80b3b548a4b5ab68470cb45517378" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b05f075731b43a88f0c881bf3c91b8a"
I20260809 04:46:47.926543 27361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7f2ed9b8d2b4d328d87635437e08233" candidate_uuid: "94f80b3b548a4b5ab68470cb45517378" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47e6dbc35e144f3292038117abd975f4"
I20260809 04:46:47.926724 27508 raft_consensus.cc:3037] T d7f2ed9b8d2b4d328d87635437e08233 P 9b05f075731b43a88f0c881bf3c91b8a [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:46:47.926724 27361 raft_consensus.cc:3037] T d7f2ed9b8d2b4d328d87635437e08233 P 47e6dbc35e144f3292038117abd975f4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:46:47.929724 27361 raft_consensus.cc:2454] T d7f2ed9b8d2b4d328d87635437e08233 P 47e6dbc35e144f3292038117abd975f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94f80b3b548a4b5ab68470cb45517378 in term 1.
I20260809 04:46:47.929721 27508 raft_consensus.cc:2454] T d7f2ed9b8d2b4d328d87635437e08233 P 9b05f075731b43a88f0c881bf3c91b8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94f80b3b548a4b5ab68470cb45517378 in term 1.
I20260809 04:46:47.930161 27127 leader_election.cc:302] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47e6dbc35e144f3292038117abd975f4, 94f80b3b548a4b5ab68470cb45517378; no voters:
I20260809 04:46:47.930469 27600 raft_consensus.cc:2781] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:46:47.931088 27600 raft_consensus.cc:686] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378 [term 1 LEADER]: Becoming Leader. State: Replica: 94f80b3b548a4b5ab68470cb45517378, State: Running, Role: LEADER
I20260809 04:46:47.931610 27600 consensus_queue.cc:237] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f80b3b548a4b5ab68470cb45517378" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 37209 } } peers { permanent_uuid: "9b05f075731b43a88f0c881bf3c91b8a" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33673 } } peers { permanent_uuid: "47e6dbc35e144f3292038117abd975f4" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 32789 } }
I20260809 04:46:47.936192 27032 catalog_manager.cc:5495] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378 reported cstate change: term changed from 0 to 1, leader changed from <none> to 94f80b3b548a4b5ab68470cb45517378 (127.12.234.1). New cstate: current_term: 1 leader_uuid: "94f80b3b548a4b5ab68470cb45517378" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f80b3b548a4b5ab68470cb45517378" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 37209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b05f075731b43a88f0c881bf3c91b8a" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47e6dbc35e144f3292038117abd975f4" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 32789 } health_report { overall_health: UNKNOWN } } }
I20260809 04:46:47.979688 27578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:47.974399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33476 (local address 127.12.234.1:37209)
0809 04:46:47.974597 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:47.974603 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:47.974622 (+ 19us) server_negotiation.cc:407] Connection header received
0809 04:46:47.974702 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:47.974707 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:47.974796 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:47.974937 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:46:47.975267 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.976321 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.977535 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:47.978074 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:47.978172 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:47.978550 (+ 378us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:46:47.979124 (+ 574us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:46:47.979255 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:47.979366 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 04:46:47.979442 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 04:46:47.987622 27361 raft_consensus.cc:1261] T d7f2ed9b8d2b4d328d87635437e08233 P 47e6dbc35e144f3292038117abd975f4 [term 1 FOLLOWER]: Refusing update from remote peer 94f80b3b548a4b5ab68470cb45517378: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:46:47.987627 27508 raft_consensus.cc:1261] T d7f2ed9b8d2b4d328d87635437e08233 P 9b05f075731b43a88f0c881bf3c91b8a [term 1 FOLLOWER]: Refusing update from remote peer 94f80b3b548a4b5ab68470cb45517378: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:46:47.988483 27605 consensus_queue.cc:1035] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47e6dbc35e144f3292038117abd975f4" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 32789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:46:47.988803 27600 consensus_queue.cc:1035] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b05f075731b43a88f0c881bf3c91b8a" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 33673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:46:48.011256 27031 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:46:48.011507 27031 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:46:48.014765 27031 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c5d11477f1404dce8742781cea6893c3: Sending DeleteTablet for 3 replicas of tablet d7f2ed9b8d2b4d328d87635437e08233
I20260809 04:46:48.015846 27177 tablet_service.cc:1514] Processing DeleteTablet for tablet d7f2ed9b8d2b4d328d87635437e08233 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:46:48 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:33470
I20260809 04:46:48.016007 27484 tablet_service.cc:1514] Processing DeleteTablet for tablet d7f2ed9b8d2b4d328d87635437e08233 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:46:48 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:43114
I20260809 04:46:48.016213 27336 tablet_service.cc:1514] Processing DeleteTablet for tablet d7f2ed9b8d2b4d328d87635437e08233 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:46:48 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:43436
I20260809 04:46:48.020215 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 27071
I20260809 04:46:48.026919 27620 tablet_replica.cc:330] T d7f2ed9b8d2b4d328d87635437e08233 P 47e6dbc35e144f3292038117abd975f4: stopping tablet replica
I20260809 04:46:48.027419 27620 raft_consensus.cc:2229] T d7f2ed9b8d2b4d328d87635437e08233 P 47e6dbc35e144f3292038117abd975f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:46:48.027930 27620 raft_consensus.cc:2258] T d7f2ed9b8d2b4d328d87635437e08233 P 47e6dbc35e144f3292038117abd975f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:46:48.028525 27619 tablet_replica.cc:330] T d7f2ed9b8d2b4d328d87635437e08233 P 9b05f075731b43a88f0c881bf3c91b8a: stopping tablet replica
I20260809 04:46:48.029218 27619 raft_consensus.cc:2229] T d7f2ed9b8d2b4d328d87635437e08233 P 9b05f075731b43a88f0c881bf3c91b8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:46:48.029716 27619 raft_consensus.cc:2258] T d7f2ed9b8d2b4d328d87635437e08233 P 9b05f075731b43a88f0c881bf3c91b8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:46:48.031823 27619 ts_tablet_manager.cc:1905] T d7f2ed9b8d2b4d328d87635437e08233 P 9b05f075731b43a88f0c881bf3c91b8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:46:48.033926 27620 ts_tablet_manager.cc:1905] T d7f2ed9b8d2b4d328d87635437e08233 P 47e6dbc35e144f3292038117abd975f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:46:48.038656 27619 ts_tablet_manager.cc:1918] T d7f2ed9b8d2b4d328d87635437e08233 P 9b05f075731b43a88f0c881bf3c91b8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:46:48.038870 27619 log.cc:1196] T d7f2ed9b8d2b4d328d87635437e08233 P 9b05f075731b43a88f0c881bf3c91b8a: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-2/wal/wals/d7f2ed9b8d2b4d328d87635437e08233
I20260809 04:46:48.039494 27619 ts_tablet_manager.cc:1939] T d7f2ed9b8d2b4d328d87635437e08233 P 9b05f075731b43a88f0c881bf3c91b8a: Deleting consensus metadata
I20260809 04:46:48.041267 27012 catalog_manager.cc:4841] TS 9b05f075731b43a88f0c881bf3c91b8a (127.12.234.3:33673): tablet d7f2ed9b8d2b4d328d87635437e08233 (table test-table [id=e6f0670652374eb884cdbbc044f6a16e]) successfully deleted
I20260809 04:46:48.048568 27618 tablet_replica.cc:330] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378: stopping tablet replica
I20260809 04:46:48.048900 27618 raft_consensus.cc:2229] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:46:48.049678 27618 raft_consensus.cc:2258] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:46:48.050184 27620 ts_tablet_manager.cc:1918] T d7f2ed9b8d2b4d328d87635437e08233 P 47e6dbc35e144f3292038117abd975f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:46:48.050379 27620 log.cc:1196] T d7f2ed9b8d2b4d328d87635437e08233 P 47e6dbc35e144f3292038117abd975f4: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786250603326224-13224-0/minicluster-data/ts-1/wal/wals/d7f2ed9b8d2b4d328d87635437e08233
I20260809 04:46:48.050856 27620 ts_tablet_manager.cc:1939] T d7f2ed9b8d2b4d328d87635437e08233 P 47e6dbc35e144f3292038117abd975f4: Deleting consensus metadata
I20260809 04:46:48.052477 27013 catalog_manager.cc:4841] TS 47e6dbc35e144f3292038117abd975f4 (127.12.234.2:32789): tablet d7f2ed9b8d2b4d328d87635437e08233 (table test-table [id=e6f0670652374eb884cdbbc044f6a16e]) successfully deleted
I20260809 04:46:48.053395 27618 ts_tablet_manager.cc:1905] T d7f2ed9b8d2b4d328d87635437e08233 P 94f80b3b548a4b5ab68470cb45517378: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:46:48.055985 27578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:48.022039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33482 (local address 127.12.234.1:37209)
0809 04:46:48.022598 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:48.022608 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:48.038681 (+ 16073us) server_negotiation.cc:407] Connection header received
0809 04:46:48.039150 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:48.039159 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:48.039270 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:48.039369 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:48.041851 (+ 2482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.043038 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.044775 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.045417 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.045517 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:48.049120 (+ 3603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:48.049135 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:48.049151 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:48.049198 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:48.053052 (+ 3854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:48.053830 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:48.053840 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:48.053860 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:48.054011 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:48.054382 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:48.054387 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:48.054392 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:48.054758 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:48.055063 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:48.055392 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 04:46:48.055620 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394}
I20260809 04:46:48.056617 27243 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:46:48.149084 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 27071
I20260809 04:46:48.170193 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 27252
W20260809 04:46:48.170177 27014 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.234.1:37209 (error 108)
I20260809 04:46:48.172749 27624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:48.171887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:37209 (local address 127.0.0.1:33490)
0809 04:46:48.172352 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:48.172492 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:37209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":213,"threads_started":1}
W20260809 04:46:48.172969 27014 catalog_manager.cc:4584] TS 94f80b3b548a4b5ab68470cb45517378 (127.12.234.1:37209): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d7f2ed9b8d2b4d328d87635437e08233: Network error: Client connection negotiation failed: client connection to 127.12.234.1:37209: connect: Connection refused (error 111)
I20260809 04:46:48.183421 27582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:48.170975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43452 (local address 127.12.234.2:32789)
0809 04:46:48.171145 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:48.171150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:48.171167 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:46:48.171404 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:48.171408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:48.171468 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:48.171590 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:48.173315 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.174018 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.175156 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.175585 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.175682 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:48.177984 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:48.177995 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:48.178008 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:48.178042 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:48.180931 (+ 2889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:48.181576 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:48.181583 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:48.181586 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:48.181734 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:48.182102 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:48.182105 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:48.182107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:48.182482 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:48.182755 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:48.182973 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 04:46:48.183146 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 04:46:48.184190 27403 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:46:48.226626 27624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:48.226234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:37209 (local address 127.0.0.1:33506)
0809 04:46:48.226407 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:48.226496 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:37209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 04:46:48.227669 27624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:48.227305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:37209 (local address 127.0.0.1:33510)
0809 04:46:48.227443 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:48.227529 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:37209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 04:46:48.272323 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 27252
I20260809 04:46:48.288349 27624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:48.287730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:37209 (local address 127.0.0.1:33518)
0809 04:46:48.287994 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:48.288124 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:37209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 04:46:48.289458 27624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:48.289014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:37209 (local address 127.0.0.1:33520)
0809 04:46:48.289195 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:48.289280 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:37209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 04:46:48.293696 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 27412
I20260809 04:46:48.307420 27580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:48.294438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43116 (local address 127.12.234.3:33673)
0809 04:46:48.294613 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:48.294618 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:48.294637 (+ 19us) server_negotiation.cc:407] Connection header received
0809 04:46:48.294697 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:48.294702 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:48.294768 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:48.294872 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:48.296047 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.297179 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.298330 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:48.298912 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:48.299024 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:48.301514 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:48.301526 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:48.301540 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:48.301579 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:48.304960 (+ 3381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:48.305541 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:48.305550 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:48.305554 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:48.305705 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:48.306031 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:48.306036 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:48.306039 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:48.306453 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:48.306713 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:48.306891 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 04:46:48.307078 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 04:46:48.308172 27552 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:46:48.369939 27624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:48.369510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:37209 (local address 127.0.0.1:33528)
0809 04:46:48.369724 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:48.369803 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:37209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 04:46:48.371057 27624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:48.370591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:37209 (local address 127.0.0.1:33544)
0809 04:46:48.370843 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:48.370936 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:37209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260809 04:46:48.390398 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 27412
I20260809 04:46:48.409863 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 26989
I20260809 04:46:48.410943 27064 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:46:48.515249 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 26989
I20260809 04:46:48.542029 13224 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 04:46:48.886382 13224 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 04:46:48.886606 13224 mini_ranger.cc:66] Stopping Ranger...
I20260809 04:46:49.230784 13224 mini_ranger.cc:68] Stopped Ranger
2026-08-09 04:46:49.231 UTC [25927] LOG: received smart shutdown request
2026-08-09 04:46:49.234 UTC [25927] LOG: background worker "logical replication launcher" (PID 25938) exited with exit code 1
2026-08-09 04:46:49.234 UTC [25931] LOG: shutting down
2026-08-09 04:46:49.382 UTC [25927] LOG: database system is shut down
2026-08-09T04:46:49Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (53964 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:46:49 dist-test-slave-1jjb krb5kdc[27636](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:46:49 dist-test-slave-1jjb krb5kdc[27636](info): set up 2 sockets
Aug 09 04:46:49 dist-test-slave-1jjb krb5kdc[27636](info): commencing operation
krb5kdc: starting...
W20260809 04:46:50.599354 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.002s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:46:50 dist-test-slave-1jjb krb5kdc[27636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:46:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:46:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:46:50.822100 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:36739
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:44335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:36739,127.12.234.61:34033,127.12.234.60:38085
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:46:51.131484 27650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:46:51.131791 27650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:46:51.131963 27650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:46:51.138693 27650 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:46:51.138818 27650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:46:51.138912 27650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:46:51.138990 27650 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:46:51.139050 27650 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:46:51.147775 27650 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:44335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:36739,127.12.234.61:34033,127.12.234.60:38085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:36739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:46:51.148917 27650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:46:51.150115 27650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:46:51.160586 27657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:51.160640 27656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:51.160526 27659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:51.161300 27650 server_base.cc:1034] running on GCE node
I20260809 04:46:51.161980 27650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:46:51.163131 27650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:46:51.164399 27650 hybrid_clock.cc:648] HybridClock initialized: now 1786250811164315 us; error 67 us; skew 500 ppm
I20260809 04:46:51.164744 27650 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:46:51 dist-test-slave-1jjb krb5kdc[27636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250811, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:46:51.167652 27650 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:46:51.168999 27650 webserver.cc:460] Webserver started at http://127.12.234.62:46191/ using document root <none> and password file <none>
I20260809 04:46:51.169718 27650 fs_manager.cc:359] Metadata directory not provided
I20260809 04:46:51.169878 27650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:46:51.170210 27650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:46:51.173094 27650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "a10ac0d39e1446358a551e7b57dd7121"
format_stamp: "Formatted at 2026-08-09 04:46:51 on dist-test-slave-1jjb"
server_key: "5f473defce4d56cf0adff14ff61865db"
server_key_iv: "0077188afaf4267946b8eaf0cfdaf5ce"
server_key_version: "encryptionkey@0"
I20260809 04:46:51.173872 27650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "a10ac0d39e1446358a551e7b57dd7121"
format_stamp: "Formatted at 2026-08-09 04:46:51 on dist-test-slave-1jjb"
server_key: "5f473defce4d56cf0adff14ff61865db"
server_key_iv: "0077188afaf4267946b8eaf0cfdaf5ce"
server_key_version: "encryptionkey@0"
I20260809 04:46:51.178432 27650 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:46:51.181845 27669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:51.182991 27650 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 04:46:51.183197 27650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "a10ac0d39e1446358a551e7b57dd7121"
format_stamp: "Formatted at 2026-08-09 04:46:51 on dist-test-slave-1jjb"
server_key: "5f473defce4d56cf0adff14ff61865db"
server_key_iv: "0077188afaf4267946b8eaf0cfdaf5ce"
server_key_version: "encryptionkey@0"
I20260809 04:46:51.183393 27650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:46:51.201140 27650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:46:51.204515 27650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:46:51.204780 27650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:46:51.222455 27650 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:36739
I20260809 04:46:51.222523 27726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:36739 every 8 connection(s)
I20260809 04:46:51.223627 27650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:46:51.226675 27728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:46:51.231618 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 27650
I20260809 04:46:51.231815 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:46:51.232486 13224 external_mini_cluster.cc:1428] Setting key 756d17c5e4677ce520f5db65dc324ff1
I20260809 04:46:51.232534 27728 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36739 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:46:51 dist-test-slave-1jjb krb5kdc[27636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250811, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:46:51.252619 27730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.235183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59090 (local address 127.12.234.62:36739)
0809 04:46:51.236038 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:51.236057 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:51.236470 (+ 413us) server_negotiation.cc:407] Connection header received
0809 04:46:51.238248 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:51.238253 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:51.238436 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:51.238640 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:51.239821 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:51.241076 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:51.242345 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:51.242822 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:51.243051 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:51.246101 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:51.246108 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:51.246118 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:51.246147 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:51.248725 (+ 2578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:51.249407 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:51.249418 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:51.249420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:51.249559 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:51.249933 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:51.249941 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:51.249944 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:51.250368 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:51.250590 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:51.251045 (+ 455us) server_negotiation.cc:299] Negotiation successful
0809 04:46:51.251308 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":415,"threads_started":1}
I20260809 04:46:51.252652 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.235207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:36739 (local address 127.0.0.1:59090)
0809 04:46:51.236323 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:51.236363 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:51.237904 (+ 1541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:51.238649 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:51.238655 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:51.239318 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:46:51.239630 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:51.239646 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:51.241236 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:51.241241 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:51.242214 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:51.242224 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:51.243083 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:51.243088 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:51.243246 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:51.243861 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:46:51.243881 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:46:51.245900 (+ 2019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:46:51.248839 (+ 2939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:51.248851 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:51.248860 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:51.249242 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:51.249667 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:51.249673 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:51.249677 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:51.249829 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:51.250537 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:46:51.250544 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:46:51.250718 (+ 174us) client_negotiation.cc:769] Sending connection context
0809 04:46:51.250937 (+ 219us) client_negotiation.cc:240] Negotiation successful
0809 04:46:51.251466 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":949,"mutex_wait_us":113,"thread_start_us":741,"threads_started":1}
I20260809 04:46:51.255582 27728 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:46:51.256794 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.256370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:34033 (local address 127.0.0.1:53140)
0809 04:46:51.256547 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:51.256666 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
WARNING: no policy specified for kudu/127.12.234.61@KRBTEST.COM; defaulting to no policy
W20260809 04:46:51.257112 27671 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34033: connect: Connection refused (error 111)
I20260809 04:46:51.258513 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.258215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:34033 (local address 127.0.0.1:53144)
0809 04:46:51.258341 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:51.258399 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 04:46:51.258833 27728 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.61:34033: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34033: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.61.keytab.
Entry for principal kudu/127.12.234.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.61.keytab.
I20260809 04:46:51.287272 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.61:34033
--webserver_interface=127.12.234.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.61.keytab
--principal=kudu/127.12.234.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:44335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:36739,127.12.234.61:34033,127.12.234.60:38085
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:46:51.308216 27728 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 } attempt: 1
I20260809 04:46:51.309412 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.308932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:34033 (local address 127.0.0.1:33360)
0809 04:46:51.309134 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:51.309264 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 04:46:51.310431 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.310093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:34033 (local address 127.0.0.1:33374)
0809 04:46:51.310218 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:51.310282 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 04:46:51.310693 27728 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.61:34033: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34033: connect: Connection refused (error 111)
I20260809 04:46:51.378929 27728 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 } attempt: 2
I20260809 04:46:51.380313 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.379647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:34033 (local address 127.0.0.1:33382)
0809 04:46:51.379879 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:51.379975 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260809 04:46:51.381402 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.381025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:34033 (local address 127.0.0.1:33384)
0809 04:46:51.381175 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:51.381257 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 04:46:51.381634 27728 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.61:34033: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34033: connect: Connection refused (error 111)
I20260809 04:46:51.472922 27728 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 } attempt: 3
I20260809 04:46:51.474388 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.473762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:34033 (local address 127.0.0.1:33396)
0809 04:46:51.474041 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:51.474184 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260809 04:46:51.475565 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.475130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:34033 (local address 127.0.0.1:33408)
0809 04:46:51.475310 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:51.475400 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260809 04:46:51.475906 27728 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.61:34033: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34033: connect: Connection refused (error 111)
W20260809 04:46:51.595134 27735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:46:51.595438 27735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:46:51.595582 27735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:46:51.602133 27735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:46:51.602273 27735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:46:51.602380 27735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:46:51.602459 27735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:46:51.602520 27735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:46:51.611543 27735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:44335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:36739,127.12.234.61:34033,127.12.234.60:38085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.61.keytab
--principal=kudu/127.12.234.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.61:34033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.234.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:46:51.612694 27735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:46:51.613864 27735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:46:51.619161 27728 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 } attempt: 4
I20260809 04:46:51.620520 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.619911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:34033 (local address 127.0.0.1:33422)
0809 04:46:51.620230 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:51.620335 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260809 04:46:51.621558 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.621119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:34033 (local address 127.0.0.1:33426)
0809 04:46:51.621260 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:51.621409 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 04:46:51.621738 27728 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.61:34033: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34033: connect: Connection refused (error 111)
W20260809 04:46:51.625712 27743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:51.625718 27741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:51.626283 27735 server_base.cc:1034] running on GCE node
W20260809 04:46:51.625921 27740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:51.627086 27735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:46:51.628409 27735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:46:51.629629 27735 hybrid_clock.cc:648] HybridClock initialized: now 1786250811629571 us; error 63 us; skew 500 ppm
I20260809 04:46:51.629964 27735 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:46:51 dist-test-slave-1jjb krb5kdc[27636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250811, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:46:51.637719 27735 init.cc:377] Logged in from keytab as kudu/127.12.234.61@KRBTEST.COM (short username kudu)
I20260809 04:46:51.639122 27735 webserver.cc:460] Webserver started at http://127.12.234.61:39785/ using document root <none> and password file <none>
I20260809 04:46:51.639786 27735 fs_manager.cc:359] Metadata directory not provided
I20260809 04:46:51.639909 27735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:46:51.640247 27735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:46:51.643217 27735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-1/data/instance:
uuid: "15154f37cf344fb38bae8755890a4051"
format_stamp: "Formatted at 2026-08-09 04:46:51 on dist-test-slave-1jjb"
server_key: "2d87ba530aa6c4ff69779e41bd9abedb"
server_key_iv: "46896c7f80f142e82c0566a3921dfc3f"
server_key_version: "encryptionkey@0"
I20260809 04:46:51.644022 27735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-1/wal/instance:
uuid: "15154f37cf344fb38bae8755890a4051"
format_stamp: "Formatted at 2026-08-09 04:46:51 on dist-test-slave-1jjb"
server_key: "2d87ba530aa6c4ff69779e41bd9abedb"
server_key_iv: "46896c7f80f142e82c0566a3921dfc3f"
server_key_version: "encryptionkey@0"
I20260809 04:46:51.648931 27735 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 04:46:51.652367 27751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:51.653554 27735 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:46:51.653735 27735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-1/wal
uuid: "15154f37cf344fb38bae8755890a4051"
format_stamp: "Formatted at 2026-08-09 04:46:51 on dist-test-slave-1jjb"
server_key: "2d87ba530aa6c4ff69779e41bd9abedb"
server_key_iv: "46896c7f80f142e82c0566a3921dfc3f"
server_key_version: "encryptionkey@0"
I20260809 04:46:51.653929 27735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:46:51.682188 27735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:46:51.685961 27735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:46:51.686257 27735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:46:51.704517 27735 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.61:34033
I20260809 04:46:51.704600 27810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.61:34033 every 8 connection(s)
I20260809 04:46:51.705696 27735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-1/data/info.pb
I20260809 04:46:51.707645 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 27735
I20260809 04:46:51.707849 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-1/wal/instance
I20260809 04:46:51.708508 13224 external_mini_cluster.cc:1428] Setting key 07ad9079208ceed5435db46b97b094f1
I20260809 04:46:51.710039 27812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:46:51.720096 27812 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36739 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:46:51 dist-test-slave-1jjb krb5kdc[27636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250811, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.61@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.234.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.60@KRBTEST.COM" created.
I20260809 04:46:51.742561 27730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.722505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39038 (local address 127.12.234.62:36739)
0809 04:46:51.722735 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:51.722743 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:51.723903 (+ 1160us) server_negotiation.cc:407] Connection header received
0809 04:46:51.726785 (+ 2882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:51.726789 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:51.726862 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:51.726945 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:51.728977 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:51.729667 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:51.732044 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:51.732482 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:51.732577 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:51.736508 (+ 3931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:51.736512 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:51.736515 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:51.736545 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:51.739169 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:51.740063 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:51.740068 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:51.740103 (+ 35us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:51.740267 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:51.740699 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:51.740706 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:51.740712 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:51.741107 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:51.741258 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:51.741894 (+ 636us) server_negotiation.cc:299] Negotiation successful
0809 04:46:51.742153 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 04:46:51.742978 27815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.722901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:36739 (local address 127.0.0.1:39038)
0809 04:46:51.723699 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:51.723771 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:51.726500 (+ 2729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:51.727188 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:51.727202 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:51.728017 (+ 815us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:46:51.728801 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:51.728826 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:51.729834 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:51.729840 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:51.731832 (+ 1992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:51.731848 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:51.732703 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:51.732713 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:51.733003 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:51.733648 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:46:51.733681 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:46:51.736303 (+ 2622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:46:51.739322 (+ 3019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:51.739327 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:51.739389 (+ 62us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:51.739900 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:51.740386 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:51.740389 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:51.740392 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:51.740555 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:51.741232 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:46:51.741238 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:46:51.741532 (+ 294us) client_negotiation.cc:769] Sending connection context
0809 04:46:51.741729 (+ 197us) client_negotiation.cc:240] Negotiation successful
0809 04:46:51.742042 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":300,"threads_started":1}
I20260809 04:46:51.744606 27812 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:46:51 dist-test-slave-1jjb krb5kdc[27636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250811, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.61@KRBTEST.COM for kudu/127.12.234.61@KRBTEST.COM
I20260809 04:46:51.760957 27815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.745373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:34033 (local address 127.0.0.1:33438)
0809 04:46:51.745508 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:51.745526 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:51.745614 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:51.746671 (+ 1057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:51.746674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:51.746985 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:46:51.747099 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:51.747105 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:51.748678 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:51.748683 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:51.750219 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:51.750237 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:51.751063 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:51.751071 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:51.751330 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:51.751901 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:46:51.751927 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:46:51.754145 (+ 2218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:46:51.758419 (+ 4274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:51.758423 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:51.758425 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:51.758828 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:51.759237 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:51.759242 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:51.759246 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:51.759472 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:51.760236 (+ 764us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:46:51.760239 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:46:51.760370 (+ 131us) client_negotiation.cc:769] Sending connection context
0809 04:46:51.760497 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 04:46:51.760688 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 04:46:51.761438 27816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.745631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33438 (local address 127.12.234.61:34033)
0809 04:46:51.746263 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:51.746272 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:51.746301 (+ 29us) server_negotiation.cc:407] Connection header received
0809 04:46:51.746349 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:51.746357 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:51.746533 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:51.746666 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:51.747245 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:51.748558 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:51.750367 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:51.750915 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:51.751007 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:51.754367 (+ 3360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:51.754380 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:51.754396 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:51.754432 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:51.758278 (+ 3846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:51.758950 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:51.758960 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:51.758966 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:51.759111 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:51.759596 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:51.759603 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:51.759608 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:51.760106 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:51.760333 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:51.760695 (+ 362us) server_negotiation.cc:299] Negotiation successful
0809 04:46:51.760985 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":328,"threads_started":1}
I20260809 04:46:51.763871 27812 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:46:51.764977 27815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.764538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:38085 (local address 127.0.0.1:49648)
0809 04:46:51.764673 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:51.764782 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 04:46:51.765239 27752 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
I20260809 04:46:51.766752 27815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.766473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:38085 (local address 127.0.0.1:49650)
0809 04:46:51.766592 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:51.766663 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 04:46:51.767006 27812 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.60:38085: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.60.keytab.
Entry for principal kudu/127.12.234.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.60.keytab.
I20260809 04:46:51.771159 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.60:38085
--webserver_interface=127.12.234.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.60.keytab
--principal=kudu/127.12.234.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:44335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:36739,127.12.234.61:34033,127.12.234.60:38085
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:46:51.816337 27812 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } attempt: 1
I20260809 04:46:51.817487 27815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.817069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:38085 (local address 127.0.0.1:49662)
0809 04:46:51.817254 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:51.817348 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 04:46:51.818634 27815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.818271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:38085 (local address 127.0.0.1:49674)
0809 04:46:51.818446 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:51.818530 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260809 04:46:51.819602 27812 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.60:38085: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
I20260809 04:46:51.887810 27812 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } attempt: 2
I20260809 04:46:51.888964 27815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.888547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:38085 (local address 127.0.0.1:49684)
0809 04:46:51.888725 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:51.888809 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 04:46:51.890046 27815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.889624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:38085 (local address 127.0.0.1:49694)
0809 04:46:51.889793 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:51.889897 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 04:46:51.890297 27812 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.60:38085: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
I20260809 04:46:51.921025 27728 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 } attempt: 5
Aug 09 04:46:51 dist-test-slave-1jjb krb5kdc[27636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250811, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for kudu/127.12.234.61@KRBTEST.COM
I20260809 04:46:51.934263 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.921760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:34033 (local address 127.0.0.1:33444)
0809 04:46:51.921966 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:51.921985 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:51.922138 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:51.922507 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:51.922513 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:51.922925 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:46:51.923112 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:51.923121 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:51.924088 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:51.924094 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:51.924903 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:51.924915 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:51.925646 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:51.925652 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:51.925822 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:51.926527 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:46:51.926545 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:46:51.928524 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:46:51.931824 (+ 3300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:51.931829 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:51.931833 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:51.932244 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:51.932619 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:51.932623 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:51.932627 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:51.932764 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:51.933280 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:46:51.933285 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:46:51.933454 (+ 169us) client_negotiation.cc:769] Sending connection context
0809 04:46:51.933591 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 04:46:51.933891 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 04:46:51.934266 27816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.921801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33444 (local address 127.12.234.61:34033)
0809 04:46:51.921985 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:51.921991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:51.922062 (+ 71us) server_negotiation.cc:407] Connection header received
0809 04:46:51.922277 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:51.922282 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:51.922362 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:51.922465 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:51.923273 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:51.923940 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:51.925048 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:51.925481 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:51.925618 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:51.928703 (+ 3085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:51.928707 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:51.928710 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:51.928740 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:51.931690 (+ 2950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:51.932359 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:51.932365 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:51.932368 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:51.932507 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:51.932887 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:51.932892 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:51.932895 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:51.933161 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:51.933315 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:51.933738 (+ 423us) server_negotiation.cc:299] Negotiation successful
0809 04:46:51.933955 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 04:46:51.935663 27728 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:46:51.936798 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.936408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:38085 (local address 127.0.0.1:49696)
0809 04:46:51.936571 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:51.936651 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 04:46:51.937695 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.937390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:38085 (local address 127.0.0.1:49704)
0809 04:46:51.937519 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:51.937609 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 04:46:51.937877 27728 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.60:38085: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
I20260809 04:46:51.981524 27812 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } attempt: 3
I20260809 04:46:51.982741 27815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.982155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:38085 (local address 127.0.0.1:49706)
0809 04:46:51.982394 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:51.982518 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 04:46:51.983980 27815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.983504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:38085 (local address 127.0.0.1:49720)
0809 04:46:51.983741 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:51.983832 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
W20260809 04:46:51.984220 27812 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.60:38085: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
I20260809 04:46:51.989090 27728 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } attempt: 1
I20260809 04:46:51.989984 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.989584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:38085 (local address 127.0.0.1:49726)
0809 04:46:51.989748 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:51.989842 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 04:46:51.990935 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:51.990616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:38085 (local address 127.0.0.1:49736)
0809 04:46:51.990760 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:51.990838 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 04:46:51.991132 27728 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.60:38085: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
I20260809 04:46:52.059324 27728 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } attempt: 2
I20260809 04:46:52.060431 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:52.059948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:38085 (local address 127.0.0.1:49742)
0809 04:46:52.060189 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:52.060280 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 04:46:52.061342 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:52.061019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:38085 (local address 127.0.0.1:49754)
0809 04:46:52.061168 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:52.061246 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 04:46:52.061533 27728 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.60:38085: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
W20260809 04:46:52.089241 27819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:46:52.089546 27819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:46:52.089696 27819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:46:52.096452 27819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:46:52.096576 27819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:46:52.096681 27819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:46:52.096774 27819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:46:52.096853 27819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:46:52.105767 27819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:44335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:36739,127.12.234.61:34033,127.12.234.60:38085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.60.keytab
--principal=kudu/127.12.234.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.60:38085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.234.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:46:52.106851 27819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:46:52.108012 27819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:46:52.119688 27825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:52.119724 27826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:52.119750 27829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:52.121107 27819 server_base.cc:1034] running on GCE node
I20260809 04:46:52.121781 27819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:46:52.122936 27819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:46:52.124168 27819 hybrid_clock.cc:648] HybridClock initialized: now 1786250812124135 us; error 44 us; skew 500 ppm
I20260809 04:46:52.124488 27819 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:46:52 dist-test-slave-1jjb krb5kdc[27636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250812, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:46:52.127456 27812 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } attempt: 4
I20260809 04:46:52.128141 27819 init.cc:377] Logged in from keytab as kudu/127.12.234.60@KRBTEST.COM (short username kudu)
I20260809 04:46:52.129029 27815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:52.128348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:38085 (local address 127.0.0.1:49758)
0809 04:46:52.128697 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:52.128823 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260809 04:46:52.129781 27819 webserver.cc:460] Webserver started at http://127.12.234.60:45005/ using document root <none> and password file <none>
I20260809 04:46:52.130126 27815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:52.129779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:38085 (local address 127.0.0.1:49768)
0809 04:46:52.129942 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:52.130026 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 04:46:52.130378 27812 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.60:38085: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
I20260809 04:46:52.130519 27819 fs_manager.cc:359] Metadata directory not provided
I20260809 04:46:52.130663 27819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:46:52.130995 27819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:46:52.133796 27819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-2/data/instance:
uuid: "166ed7f8ebcc4865a4af02ca96576053"
format_stamp: "Formatted at 2026-08-09 04:46:52 on dist-test-slave-1jjb"
server_key: "8498247b2f7aa7d75adedebb4cec8509"
server_key_iv: "68be8f543ba19d962a7c5f12b2840706"
server_key_version: "encryptionkey@0"
I20260809 04:46:52.134545 27819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-2/wal/instance:
uuid: "166ed7f8ebcc4865a4af02ca96576053"
format_stamp: "Formatted at 2026-08-09 04:46:52 on dist-test-slave-1jjb"
server_key: "8498247b2f7aa7d75adedebb4cec8509"
server_key_iv: "68be8f543ba19d962a7c5f12b2840706"
server_key_version: "encryptionkey@0"
I20260809 04:46:52.139093 27819 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 04:46:52.142539 27836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:52.143677 27819 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 04:46:52.143837 27819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-2/wal
uuid: "166ed7f8ebcc4865a4af02ca96576053"
format_stamp: "Formatted at 2026-08-09 04:46:52 on dist-test-slave-1jjb"
server_key: "8498247b2f7aa7d75adedebb4cec8509"
server_key_iv: "68be8f543ba19d962a7c5f12b2840706"
server_key_version: "encryptionkey@0"
I20260809 04:46:52.144021 27819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:46:52.157692 27819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:46:52.161234 27819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:46:52.161466 27819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:46:52.167724 27728 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } attempt: 3
I20260809 04:46:52.168850 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:52.168287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:38085 (local address 127.0.0.1:49784)
0809 04:46:52.168481 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:52.168638 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 04:46:52.170035 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:52.169674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:38085 (local address 127.0.0.1:49792)
0809 04:46:52.169855 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:52.169937 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260809 04:46:52.170279 27728 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.60:38085: Network error: Client connection negotiation failed: client connection to 127.12.234.60:38085: connect: Connection refused (error 111)
I20260809 04:46:52.178779 27819 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.60:38085
I20260809 04:46:52.178848 27896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.60:38085 every 8 connection(s)
I20260809 04:46:52.180008 27819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-2/data/info.pb
I20260809 04:46:52.181705 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 27819
I20260809 04:46:52.181895 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786250603326224-13224-0/minicluster-data/master-2/wal/instance
I20260809 04:46:52.182341 13224 external_mini_cluster.cc:1428] Setting key aeb20e5105508dfd70f4f49166c6af23
I20260809 04:46:52.183890 27897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:46:52.193050 27897 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36739 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:46:52 dist-test-slave-1jjb krb5kdc[27636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.60@KRBTEST.COM
Aug 09 04:46:52 dist-test-slave-1jjb krb5kdc[27636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250812, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.60@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:46:52.205415 27902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:52.184053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49804 (local address 127.12.234.60:38085)
0809 04:46:52.184860 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:52.184874 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:52.184919 (+ 45us) server_negotiation.cc:407] Connection header received
0809 04:46:52.186954 (+ 2035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:52.186980 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:52.187462 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:52.187850 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:52.189351 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.191380 (+ 2029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.192724 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.193734 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.193912 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:52.198491 (+ 4579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:52.198511 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:52.198520 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:52.198551 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:52.201409 (+ 2858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:52.202101 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:52.202107 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:52.202113 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:52.202285 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:52.202688 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:52.202692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:52.202694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:52.203348 (+ 654us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:52.203670 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:52.204067 (+ 397us) server_negotiation.cc:299] Negotiation successful
0809 04:46:52.204456 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":328,"threads_started":1}
I20260809 04:46:52.210512 27730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:52.195791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39044 (local address 127.12.234.62:36739)
0809 04:46:52.196011 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:52.196017 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:52.197003 (+ 986us) server_negotiation.cc:407] Connection header received
0809 04:46:52.197220 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:52.197224 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:52.197284 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:52.197390 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:52.198417 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.199063 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.200565 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.200969 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.201101 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:52.204399 (+ 3298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:52.204404 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:52.204407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:52.204436 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:52.207346 (+ 2910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:52.208216 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:52.208220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:52.208222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:52.208416 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:52.208903 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:52.208907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:52.208909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:52.209155 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:52.209346 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:52.209861 (+ 515us) server_negotiation.cc:299] Negotiation successful
0809 04:46:52.210091 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 04:46:52.210816 27903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:52.195794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:36739 (local address 127.0.0.1:39044)
0809 04:46:52.196855 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:52.196927 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:52.197082 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:52.197443 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:52.197448 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:52.198013 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:46:52.198264 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:52.198282 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.199205 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.199210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:52.200405 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:52.200417 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.201160 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.201170 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:52.201337 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:52.201913 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:46:52.201928 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:46:52.204245 (+ 2317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:46:52.207466 (+ 3221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:52.207494 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:52.207502 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:52.208055 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:52.208515 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:52.208519 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:52.208522 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:52.208765 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:52.209285 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:46:52.209292 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:46:52.209515 (+ 223us) client_negotiation.cc:769] Sending connection context
0809 04:46:52.209704 (+ 189us) client_negotiation.cc:240] Negotiation successful
0809 04:46:52.210086 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"thread_start_us":499,"threads_started":1}
I20260809 04:46:52.212059 27897 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:46:52 dist-test-slave-1jjb krb5kdc[27636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
Aug 09 04:46:52 dist-test-slave-1jjb krb5kdc[27636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250812, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.60@KRBTEST.COM for kudu/127.12.234.61@KRBTEST.COM
I20260809 04:46:52.223707 27904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:52.208447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39052 (local address 127.12.234.62:36739)
0809 04:46:52.209032 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:52.209036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:52.209065 (+ 29us) server_negotiation.cc:407] Connection header received
0809 04:46:52.209102 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:52.209105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:52.209152 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:52.209265 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:52.210803 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.211794 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.213389 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.213777 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.213907 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:52.217855 (+ 3948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:52.217864 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:52.217871 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:52.217911 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:52.220699 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:52.221452 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:52.221456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:52.221458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:52.221570 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:52.221994 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:52.221998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:52.222001 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:52.222326 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:52.222587 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:52.223091 (+ 504us) server_negotiation.cc:299] Negotiation successful
0809 04:46:52.223300 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":291,"threads_started":1}
I20260809 04:46:52.224915 27903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:52.212842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:34033 (local address 127.0.0.1:33450)
0809 04:46:52.212995 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:52.213011 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:52.213115 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:52.213403 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:52.213407 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:52.213748 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:46:52.213879 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:52.213887 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.214931 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.214938 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:52.215738 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:52.215749 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.216504 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.216508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:52.216631 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:52.217144 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:46:52.217158 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:46:52.219034 (+ 1876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:46:52.222428 (+ 3394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:52.222433 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:52.222437 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:52.222765 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:52.223174 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:52.223178 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:52.223182 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:52.223345 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:52.224008 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:46:52.224011 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:46:52.224180 (+ 169us) client_negotiation.cc:769] Sending connection context
0809 04:46:52.224307 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 04:46:52.224596 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 04:46:52.225011 27816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:52.212795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33450 (local address 127.12.234.61:34033)
0809 04:46:52.212960 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:52.212966 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:52.213080 (+ 114us) server_negotiation.cc:407] Connection header received
0809 04:46:52.213227 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:52.213236 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:52.213291 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:52.213390 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:52.214033 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.214795 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.215875 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.216355 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.216489 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:52.219207 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:52.219211 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:52.219214 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:52.219244 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:52.222307 (+ 3063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:52.222883 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:52.222888 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:52.222891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:52.223048 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:52.223484 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:52.223489 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:52.223492 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:52.223806 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:52.223932 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:52.224439 (+ 507us) server_negotiation.cc:299] Negotiation successful
0809 04:46:52.224648 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:46:52.225884 27897 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:46:52 dist-test-slave-1jjb krb5kdc[27636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250812, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.60@KRBTEST.COM for kudu/127.12.234.60@KRBTEST.COM
Aug 09 04:46:52 dist-test-slave-1jjb krb5kdc[27636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.61@KRBTEST.COM
I20260809 04:46:52.238782 27902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:52.226523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49820 (local address 127.12.234.60:38085)
0809 04:46:52.226664 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:52.226669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:52.226738 (+ 69us) server_negotiation.cc:407] Connection header received
0809 04:46:52.226891 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:52.226895 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:52.226953 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:52.227041 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:52.227870 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.228787 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.230373 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.230937 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.231065 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:52.233531 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:52.233536 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:52.233540 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:52.233572 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:52.236470 (+ 2898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:52.237108 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:52.237115 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:52.237120 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:52.237254 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:52.237571 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:52.237575 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:52.237577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:52.237890 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:52.238021 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:52.238382 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 04:46:52.238537 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 04:46:52.239125 27903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:52.226539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:38085 (local address 127.0.0.1:49820)
0809 04:46:52.226671 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:52.226687 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:52.226781 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:52.227084 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:52.227090 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:52.227538 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:46:52.227714 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:52.227727 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.228922 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.228926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:52.230223 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:52.230237 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.231079 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.231084 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:52.231278 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:52.231745 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:46:52.231759 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:46:52.233378 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:46:52.236610 (+ 3232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:52.236616 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:52.236621 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:52.236988 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:52.237364 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:52.237367 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:52.237369 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:52.237470 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:52.237991 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:46:52.237994 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:46:52.238105 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 04:46:52.238293 (+ 188us) client_negotiation.cc:240] Negotiation successful
0809 04:46:52.238711 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":102}
I20260809 04:46:52.240878 27816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:52.225770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33458 (local address 127.12.234.61:34033)
0809 04:46:52.225926 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:52.225932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:52.225948 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:46:52.226093 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:52.226098 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:52.226156 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:52.226247 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:52.228168 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.228792 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.230030 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.230552 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.230687 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:52.234965 (+ 4278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:52.234972 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:52.234978 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:52.235019 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:52.238044 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:52.238771 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:52.238778 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:52.238782 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:52.238970 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:52.239421 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:52.239425 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:52.239427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:52.239732 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:52.240016 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:52.240315 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 04:46:52.240513 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:46:52.244413 27897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053: Bootstrap starting.
I20260809 04:46:52.246049 13224 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:46:52.247822 27897 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053: Neither blocks nor log segments found. Creating new log.
I20260809 04:46:52.248939 27897 log.cc:824] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053: Log is configured to *not* fsync() on all Append() calls
I20260809 04:46:52.254253 27897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053: No bootstrap required, opened a new log
I20260809 04:46:52.270896 27897 raft_consensus.cc:348] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10ac0d39e1446358a551e7b57dd7121" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36739 } } peers { permanent_uuid: "15154f37cf344fb38bae8755890a4051" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 } } peers { permanent_uuid: "166ed7f8ebcc4865a4af02ca96576053" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } }
I20260809 04:46:52.271271 27897 raft_consensus.cc:374] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:46:52.271397 27897 raft_consensus.cc:729] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166ed7f8ebcc4865a4af02ca96576053, State: Initialized, Role: FOLLOWER
I20260809 04:46:52.272146 27897 consensus_queue.cc:260] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10ac0d39e1446358a551e7b57dd7121" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36739 } } peers { permanent_uuid: "15154f37cf344fb38bae8755890a4051" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 } } peers { permanent_uuid: "166ed7f8ebcc4865a4af02ca96576053" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } }
I20260809 04:46:52.279105 27913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10ac0d39e1446358a551e7b57dd7121" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36739 } } peers { permanent_uuid: "15154f37cf344fb38bae8755890a4051" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 } } peers { permanent_uuid: "166ed7f8ebcc4865a4af02ca96576053" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } } }
I20260809 04:46:52.279552 27913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:46:52.280290 27904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:52.251503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39064 (local address 127.12.234.62:36739)
0809 04:46:52.251667 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:52.251674 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:52.261654 (+ 9980us) server_negotiation.cc:407] Connection header received
0809 04:46:52.261838 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:52.261847 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:52.261958 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:52.262066 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:52.264983 (+ 2917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.265615 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.267877 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.268493 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.268577 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:52.272177 (+ 3600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:52.272183 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:52.272187 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:52.272222 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:52.276010 (+ 3788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:52.277127 (+ 1117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:52.277132 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:52.277136 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:52.277533 (+ 397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:52.278233 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:52.278240 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:52.278243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:52.278842 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:52.279063 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:52.279658 (+ 595us) server_negotiation.cc:299] Negotiation successful
0809 04:46:52.279913 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 04:46:52.281376 27897 sys_catalog.cc:564] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:46:52.282912 27902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:52.251057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49824 (local address 127.12.234.60:38085)
0809 04:46:52.251250 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:52.251256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:52.265511 (+ 14255us) server_negotiation.cc:407] Connection header received
0809 04:46:52.265667 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:52.265674 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:52.265737 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:52.265832 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:52.267873 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.268905 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.271398 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.272113 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.272199 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:52.275891 (+ 3692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:52.275897 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:52.275901 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:52.275936 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:52.279402 (+ 3466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:52.280225 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:52.280231 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:52.280233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:52.280450 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:52.280972 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:52.280977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:52.280980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:52.281223 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:52.281379 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:52.282396 (+ 1017us) server_negotiation.cc:299] Negotiation successful
0809 04:46:52.282594 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 04:46:52.284008 27816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:52.251474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33470 (local address 127.12.234.61:34033)
0809 04:46:52.251615 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:52.251620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:52.264827 (+ 13207us) server_negotiation.cc:407] Connection header received
0809 04:46:52.265718 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:52.265723 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:52.265940 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:52.266006 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:52.267922 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.268928 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.271505 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.272057 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.272178 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:52.276429 (+ 4251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:52.276437 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:52.276444 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:52.276501 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:52.280189 (+ 3688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:52.281389 (+ 1200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:52.281397 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:52.281401 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:52.281557 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:52.282079 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:52.282085 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:52.282088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:52.282409 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:52.282675 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:52.283351 (+ 676us) server_negotiation.cc:299] Negotiation successful
0809 04:46:52.283603 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 04:46:52.295775 27927 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:46:52.295892 27927 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:46:52.293229 27819 master_runner.cc:186] Error getting master registration for 127.12.234.62:36739: OK, instance_id { permanent_uuid: "a10ac0d39e1446358a551e7b57dd7121" instance_seqno: 1786250811217425 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 04:46:52.301661 27819 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 04:46:52.347506 27728 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } attempt: 4
Aug 09 04:46:52 dist-test-slave-1jjb krb5kdc[27636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250811, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for kudu/127.12.234.60@KRBTEST.COM
I20260809 04:46:52.362741 27902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:52.348157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49832 (local address 127.12.234.60:38085)
0809 04:46:52.348368 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:52.348375 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:52.348395 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:46:52.348498 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:52.348503 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:52.348567 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:52.348674 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:52.349564 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.350873 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.352595 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.353035 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.353139 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:52.356667 (+ 3528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:52.356671 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:52.356674 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:52.356707 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:52.359730 (+ 3023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:52.360575 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:52.360580 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:52.360583 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:52.360747 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:52.361178 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:52.361184 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:52.361188 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:52.361506 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:52.361669 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:52.362102 (+ 433us) server_negotiation.cc:299] Negotiation successful
0809 04:46:52.362339 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 04:46:52.363688 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:52.348027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:38085 (local address 127.0.0.1:49832)
0809 04:46:52.348236 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:52.348276 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:52.348393 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:52.348708 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:52.348711 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:52.349147 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:46:52.349371 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:52.349406 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.351032 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.351037 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:52.352457 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:52.352472 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.353182 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.353189 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:52.353397 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:52.353995 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:46:52.354017 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:46:52.356488 (+ 2471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:46:52.359883 (+ 3395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:52.359891 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:52.359896 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:52.360414 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:52.360883 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:52.360888 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:52.360892 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:52.361051 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:52.361651 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:46:52.361657 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:46:52.361837 (+ 180us) client_negotiation.cc:769] Sending connection context
0809 04:46:52.361981 (+ 144us) client_negotiation.cc:240] Negotiation successful
0809 04:46:52.362254 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 04:46:52.369522 27728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a10ac0d39e1446358a551e7b57dd7121: Bootstrap starting.
I20260809 04:46:52.373508 27728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a10ac0d39e1446358a551e7b57dd7121: Neither blocks nor log segments found. Creating new log.
I20260809 04:46:52.374794 27728 log.cc:824] T 00000000000000000000000000000000 P a10ac0d39e1446358a551e7b57dd7121: Log is configured to *not* fsync() on all Append() calls
I20260809 04:46:52.377940 27728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a10ac0d39e1446358a551e7b57dd7121: No bootstrap required, opened a new log
I20260809 04:46:52.388204 27728 raft_consensus.cc:348] T 00000000000000000000000000000000 P a10ac0d39e1446358a551e7b57dd7121 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10ac0d39e1446358a551e7b57dd7121" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36739 } } peers { permanent_uuid: "15154f37cf344fb38bae8755890a4051" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 } } peers { permanent_uuid: "166ed7f8ebcc4865a4af02ca96576053" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } }
I20260809 04:46:52.388562 27728 raft_consensus.cc:374] T 00000000000000000000000000000000 P a10ac0d39e1446358a551e7b57dd7121 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:46:52.388667 27728 raft_consensus.cc:729] T 00000000000000000000000000000000 P a10ac0d39e1446358a551e7b57dd7121 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a10ac0d39e1446358a551e7b57dd7121, State: Initialized, Role: FOLLOWER
I20260809 04:46:52.389269 27728 consensus_queue.cc:260] T 00000000000000000000000000000000 P a10ac0d39e1446358a551e7b57dd7121 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10ac0d39e1446358a551e7b57dd7121" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36739 } } peers { permanent_uuid: "15154f37cf344fb38bae8755890a4051" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 } } peers { permanent_uuid: "166ed7f8ebcc4865a4af02ca96576053" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } }
I20260809 04:46:52.390206 27930 sys_catalog.cc:455] T 00000000000000000000000000000000 P a10ac0d39e1446358a551e7b57dd7121 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10ac0d39e1446358a551e7b57dd7121" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36739 } } peers { permanent_uuid: "15154f37cf344fb38bae8755890a4051" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 } } peers { permanent_uuid: "166ed7f8ebcc4865a4af02ca96576053" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } } }
I20260809 04:46:52.390647 27930 sys_catalog.cc:458] T 00000000000000000000000000000000 P a10ac0d39e1446358a551e7b57dd7121 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:46:52.391422 27728 sys_catalog.cc:564] T 00000000000000000000000000000000 P a10ac0d39e1446358a551e7b57dd7121 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 04:46:52.401891 27943 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a10ac0d39e1446358a551e7b57dd7121: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:46:52.402060 27943 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:46:52.401377 27650 master_runner.cc:186] Error getting master registration for 127.12.234.61:34033: OK, instance_id { permanent_uuid: "15154f37cf344fb38bae8755890a4051" instance_seqno: 1786250811699285 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 04:46:52.412071 27650 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 04:46:52.413156 27842 tablet.cc:2369] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:46:52.429864 27812 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } attempt: 5
Aug 09 04:46:52 dist-test-slave-1jjb krb5kdc[27636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250811, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.61@KRBTEST.COM for kudu/127.12.234.60@KRBTEST.COM
I20260809 04:46:52.446388 27815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:52.430428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:38085 (local address 127.0.0.1:49840)
0809 04:46:52.430716 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:52.430736 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:52.430911 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:52.431283 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:52.431288 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:52.431831 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:46:52.432060 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:52.432072 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.433511 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.433517 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:52.434931 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:52.434950 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.435953 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.435960 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:52.436188 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:52.436886 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:46:52.436912 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:46:52.439630 (+ 2718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:46:52.443786 (+ 4156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:52.443791 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:52.443795 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:52.444319 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:52.444716 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:52.444723 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:52.444726 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:52.444870 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:52.445461 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:46:52.445466 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:46:52.445605 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 04:46:52.445747 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 04:46:52.446006 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150}
I20260809 04:46:52.446770 27902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:52.430567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49840 (local address 127.12.234.60:38085)
0809 04:46:52.430764 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:52.430770 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:52.430793 (+ 23us) server_negotiation.cc:407] Connection header received
0809 04:46:52.431048 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:52.431054 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:52.431131 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:52.431231 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:52.432250 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.433357 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.435102 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:52.435792 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:52.435919 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:52.439819 (+ 3900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:52.439825 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:52.439830 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:52.439878 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:52.443645 (+ 3767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:52.444430 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:52.444435 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:52.444439 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:52.444607 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:52.444988 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:52.444994 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:52.444997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:52.445340 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:52.445490 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:52.445921 (+ 431us) server_negotiation.cc:299] Negotiation successful
0809 04:46:52.446149 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 04:46:52.452401 27812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15154f37cf344fb38bae8755890a4051: Bootstrap starting.
I20260809 04:46:52.457314 27812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15154f37cf344fb38bae8755890a4051: Neither blocks nor log segments found. Creating new log.
W20260809 04:46:52.457401 27677 tablet.cc:2369] T 00000000000000000000000000000000 P a10ac0d39e1446358a551e7b57dd7121: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:46:52.458720 27812 log.cc:824] T 00000000000000000000000000000000 P 15154f37cf344fb38bae8755890a4051: Log is configured to *not* fsync() on all Append() calls
I20260809 04:46:52.461882 27812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15154f37cf344fb38bae8755890a4051: No bootstrap required, opened a new log
I20260809 04:46:52.473424 27812 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15154f37cf344fb38bae8755890a4051 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10ac0d39e1446358a551e7b57dd7121" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36739 } } peers { permanent_uuid: "15154f37cf344fb38bae8755890a4051" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 } } peers { permanent_uuid: "166ed7f8ebcc4865a4af02ca96576053" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } }
I20260809 04:46:52.473775 27812 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15154f37cf344fb38bae8755890a4051 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:46:52.473891 27812 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15154f37cf344fb38bae8755890a4051 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15154f37cf344fb38bae8755890a4051, State: Initialized, Role: FOLLOWER
I20260809 04:46:52.474442 27812 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15154f37cf344fb38bae8755890a4051 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10ac0d39e1446358a551e7b57dd7121" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36739 } } peers { permanent_uuid: "15154f37cf344fb38bae8755890a4051" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 } } peers { permanent_uuid: "166ed7f8ebcc4865a4af02ca96576053" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } }
I20260809 04:46:52.475310 27946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15154f37cf344fb38bae8755890a4051 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10ac0d39e1446358a551e7b57dd7121" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36739 } } peers { permanent_uuid: "15154f37cf344fb38bae8755890a4051" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 } } peers { permanent_uuid: "166ed7f8ebcc4865a4af02ca96576053" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } } }
I20260809 04:46:52.475714 27946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15154f37cf344fb38bae8755890a4051 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:46:52.476449 27812 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15154f37cf344fb38bae8755890a4051 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:46:52.493784 27735 master_runner.cc:263] No leader master found from master 15154f37cf344fb38bae8755890a4051
I20260809 04:46:52.493904 27735 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 04:46:52.496435 27957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 15154f37cf344fb38bae8755890a4051: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:46:52.496606 27957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:46:52.672544 27913 raft_consensus.cc:482] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:46:52.672878 27913 raft_consensus.cc:504] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10ac0d39e1446358a551e7b57dd7121" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36739 } } peers { permanent_uuid: "15154f37cf344fb38bae8755890a4051" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 } } peers { permanent_uuid: "166ed7f8ebcc4865a4af02ca96576053" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } }
I20260809 04:46:52.674510 27913 leader_election.cc:288] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a10ac0d39e1446358a551e7b57dd7121 (127.12.234.62:36739), 15154f37cf344fb38bae8755890a4051 (127.12.234.61:34033)
I20260809 04:46:52.675694 27781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "166ed7f8ebcc4865a4af02ca96576053" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15154f37cf344fb38bae8755890a4051" is_pre_election: true
I20260809 04:46:52.676380 27781 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 15154f37cf344fb38bae8755890a4051 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 166ed7f8ebcc4865a4af02ca96576053 in term 0.
I20260809 04:46:52.677238 27838 leader_election.cc:302] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15154f37cf344fb38bae8755890a4051, 166ed7f8ebcc4865a4af02ca96576053; no voters:
I20260809 04:46:52.677796 27913 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:46:52.677997 27913 raft_consensus.cc:482] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:46:52.678105 27913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:46:52.681085 27698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "166ed7f8ebcc4865a4af02ca96576053" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a10ac0d39e1446358a551e7b57dd7121" is_pre_election: true
I20260809 04:46:52.681663 27698 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a10ac0d39e1446358a551e7b57dd7121 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 166ed7f8ebcc4865a4af02ca96576053 in term 0.
I20260809 04:46:52.684096 27913 raft_consensus.cc:504] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10ac0d39e1446358a551e7b57dd7121" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36739 } } peers { permanent_uuid: "15154f37cf344fb38bae8755890a4051" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 } } peers { permanent_uuid: "166ed7f8ebcc4865a4af02ca96576053" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } }
I20260809 04:46:52.685231 27698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "166ed7f8ebcc4865a4af02ca96576053" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a10ac0d39e1446358a551e7b57dd7121"
I20260809 04:46:52.685433 27698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a10ac0d39e1446358a551e7b57dd7121 [term 0 FOLLOWER]: Advancing to term 1
W20260809 04:46:52.688925 27757 tablet.cc:2369] T 00000000000000000000000000000000 P 15154f37cf344fb38bae8755890a4051: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:46:52.690529 27698 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a10ac0d39e1446358a551e7b57dd7121 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 166ed7f8ebcc4865a4af02ca96576053 in term 1.
I20260809 04:46:52.691015 27839 leader_election.cc:302] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 166ed7f8ebcc4865a4af02ca96576053, a10ac0d39e1446358a551e7b57dd7121; no voters:
I20260809 04:46:52.692468 27781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "166ed7f8ebcc4865a4af02ca96576053" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15154f37cf344fb38bae8755890a4051"
I20260809 04:46:52.692693 27781 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15154f37cf344fb38bae8755890a4051 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:46:52.698519 27781 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 15154f37cf344fb38bae8755890a4051 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 166ed7f8ebcc4865a4af02ca96576053 in term 1.
I20260809 04:46:52.701099 27913 leader_election.cc:288] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [CANDIDATE]: Term 1 election: Requested vote from peers a10ac0d39e1446358a551e7b57dd7121 (127.12.234.62:36739), 15154f37cf344fb38bae8755890a4051 (127.12.234.61:34033)
I20260809 04:46:52.701464 27913 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:46:52.701866 27913 raft_consensus.cc:686] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [term 1 LEADER]: Becoming Leader. State: Replica: 166ed7f8ebcc4865a4af02ca96576053, State: Running, Role: LEADER
I20260809 04:46:52.702610 27913 consensus_queue.cc:237] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10ac0d39e1446358a551e7b57dd7121" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36739 } } peers { permanent_uuid: "15154f37cf344fb38bae8755890a4051" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 } } peers { permanent_uuid: "166ed7f8ebcc4865a4af02ca96576053" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } }
I20260809 04:46:52.733379 27960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 166ed7f8ebcc4865a4af02ca96576053. Latest consensus state: current_term: 1 leader_uuid: "166ed7f8ebcc4865a4af02ca96576053" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10ac0d39e1446358a551e7b57dd7121" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36739 } } peers { permanent_uuid: "15154f37cf344fb38bae8755890a4051" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 } } peers { permanent_uuid: "166ed7f8ebcc4865a4af02ca96576053" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } } }
I20260809 04:46:52.733798 27960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [sys.catalog]: This master's current role is: LEADER
I20260809 04:46:52.740069 27962 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:46:52.742200 27962 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:46:52.762282 27781 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 15154f37cf344fb38bae8755890a4051 [term 1 FOLLOWER]: Refusing update from remote peer 166ed7f8ebcc4865a4af02ca96576053: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:46:52.763453 27960 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15154f37cf344fb38bae8755890a4051" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:46:52.764088 27698 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a10ac0d39e1446358a551e7b57dd7121 [term 1 FOLLOWER]: Refusing update from remote peer 166ed7f8ebcc4865a4af02ca96576053: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:46:52.765007 27960 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a10ac0d39e1446358a551e7b57dd7121" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:46:52.785789 27946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15154f37cf344fb38bae8755890a4051 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 166ed7f8ebcc4865a4af02ca96576053. Latest consensus state: current_term: 1 leader_uuid: "166ed7f8ebcc4865a4af02ca96576053" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10ac0d39e1446358a551e7b57dd7121" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36739 } } peers { permanent_uuid: "15154f37cf344fb38bae8755890a4051" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 } } peers { permanent_uuid: "166ed7f8ebcc4865a4af02ca96576053" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } } }
I20260809 04:46:52.786069 27946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15154f37cf344fb38bae8755890a4051 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:46:52.789633 27963 mvcc.cc:205] Tried to move back new op lower bound from 7316483329063669760 to 7316483328840929280. Current Snapshot: MvccSnapshot[applied={T|T < 7316483329063669760}]
I20260809 04:46:52.791332 27964 mvcc.cc:205] Tried to move back new op lower bound from 7316483329063669760 to 7316483328840929280. Current Snapshot: MvccSnapshot[applied={T|T < 7316483329063669760}]
I20260809 04:46:52.799484 27913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "166ed7f8ebcc4865a4af02ca96576053" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10ac0d39e1446358a551e7b57dd7121" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36739 } } peers { permanent_uuid: "15154f37cf344fb38bae8755890a4051" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 } } peers { permanent_uuid: "166ed7f8ebcc4865a4af02ca96576053" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } } }
I20260809 04:46:52.799713 27913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [sys.catalog]: This master's current role is: LEADER
I20260809 04:46:52.800354 27962 catalog_manager.cc:1347] Generated new cluster ID: 7b67c5ab1723440a95c3dd3251a6b949
I20260809 04:46:52.800453 27962 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:46:52.802198 27930 sys_catalog.cc:455] T 00000000000000000000000000000000 P a10ac0d39e1446358a551e7b57dd7121 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 166ed7f8ebcc4865a4af02ca96576053. Latest consensus state: current_term: 1 leader_uuid: "166ed7f8ebcc4865a4af02ca96576053" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10ac0d39e1446358a551e7b57dd7121" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36739 } } peers { permanent_uuid: "15154f37cf344fb38bae8755890a4051" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 } } peers { permanent_uuid: "166ed7f8ebcc4865a4af02ca96576053" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } } }
I20260809 04:46:52.802460 27930 sys_catalog.cc:458] T 00000000000000000000000000000000 P a10ac0d39e1446358a551e7b57dd7121 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:46:52.803548 27960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "166ed7f8ebcc4865a4af02ca96576053" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10ac0d39e1446358a551e7b57dd7121" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36739 } } peers { permanent_uuid: "15154f37cf344fb38bae8755890a4051" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 } } peers { permanent_uuid: "166ed7f8ebcc4865a4af02ca96576053" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } } }
I20260809 04:46:52.803745 27960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053 [sys.catalog]: This master's current role is: LEADER
I20260809 04:46:52.806097 27946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15154f37cf344fb38bae8755890a4051 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "166ed7f8ebcc4865a4af02ca96576053" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10ac0d39e1446358a551e7b57dd7121" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36739 } } peers { permanent_uuid: "15154f37cf344fb38bae8755890a4051" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 } } peers { permanent_uuid: "166ed7f8ebcc4865a4af02ca96576053" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } } }
I20260809 04:46:52.806357 27946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15154f37cf344fb38bae8755890a4051 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:46:52.819242 27930 sys_catalog.cc:455] T 00000000000000000000000000000000 P a10ac0d39e1446358a551e7b57dd7121 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "166ed7f8ebcc4865a4af02ca96576053" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a10ac0d39e1446358a551e7b57dd7121" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36739 } } peers { permanent_uuid: "15154f37cf344fb38bae8755890a4051" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34033 } } peers { permanent_uuid: "166ed7f8ebcc4865a4af02ca96576053" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 38085 } } }
I20260809 04:46:52.819530 27930 sys_catalog.cc:458] T 00000000000000000000000000000000 P a10ac0d39e1446358a551e7b57dd7121 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:46:52.841778 27962 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:46:52.843264 27962 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:46:52.855875 27962 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 166ed7f8ebcc4865a4af02ca96576053: Generated new TSK 0
I20260809 04:46:52.857507 27962 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:46:53.404347 27943 catalog_manager.cc:1259] Loaded cluster ID: 7b67c5ab1723440a95c3dd3251a6b949
I20260809 04:46:53.404486 27943 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a10ac0d39e1446358a551e7b57dd7121: loading cluster ID for follower catalog manager: success
I20260809 04:46:53.406679 27943 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a10ac0d39e1446358a551e7b57dd7121: acquiring CA information for follower catalog manager: success
I20260809 04:46:53.408416 27943 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a10ac0d39e1446358a551e7b57dd7121: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 04:46:53.498461 27957 catalog_manager.cc:1259] Loaded cluster ID: 7b67c5ab1723440a95c3dd3251a6b949
I20260809 04:46:53.498555 27957 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 15154f37cf344fb38bae8755890a4051: loading cluster ID for follower catalog manager: success
I20260809 04:46:53.500993 27957 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 15154f37cf344fb38bae8755890a4051: acquiring CA information for follower catalog manager: success
I20260809 04:46:53.502686 27957 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 15154f37cf344fb38bae8755890a4051: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 04:46:53.954624 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 27650
I20260809 04:46:53.955804 27721 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:46:54.059055 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 27650
I20260809 04:46:54.084405 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 27735
I20260809 04:46:54.085379 27805 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:46:54.197333 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 27735
I20260809 04:46:54.220360 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 27819
I20260809 04:46:54.221522 27892 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:46:54.323320 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 27819
2026-08-09T04:46:54Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4812 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:46:54 dist-test-slave-1jjb krb5kdc[27984](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:46:54 dist-test-slave-1jjb krb5kdc[27984](info): set up 2 sockets
Aug 09 04:46:54 dist-test-slave-1jjb krb5kdc[27984](info): commencing operation
krb5kdc: starting...
W20260809 04:46:55.436714 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.001s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:46:55 dist-test-slave-1jjb krb5kdc[27984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:46:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:46:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:46:55.659698 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:42407
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:32941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:42407
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:46:55.966012 27999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:46:55.966245 27999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:46:55.966333 27999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:46:55.972685 27999 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:46:55.972772 27999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:46:55.972815 27999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:46:55.972864 27999 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:46:55.972896 27999 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:46:55.981398 27999 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:32941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:42407
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:42407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:46:55.982376 27999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:46:55.983481 27999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:46:55.994341 28005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:55.994370 28004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:55.994407 28007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:55.995201 27999 server_base.cc:1034] running on GCE node
I20260809 04:46:55.995913 27999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:46:55.997121 27999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:46:55.998361 27999 hybrid_clock.cc:648] HybridClock initialized: now 1786250815998315 us; error 40 us; skew 500 ppm
I20260809 04:46:55.998669 27999 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:46:55 dist-test-slave-1jjb krb5kdc[27984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250815, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:46:56.001605 27999 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:46:56.002995 27999 webserver.cc:460] Webserver started at http://127.12.234.62:37757/ using document root <none> and password file <none>
I20260809 04:46:56.003587 27999 fs_manager.cc:359] Metadata directory not provided
I20260809 04:46:56.003681 27999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:46:56.003904 27999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:46:56.006531 27999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "64231ecf75d0401eb83736280912fafd"
format_stamp: "Formatted at 2026-08-09 04:46:56 on dist-test-slave-1jjb"
server_key: "9ee91f2db204f69c8470e2f2e43a29d5"
server_key_iv: "c4c3bbc36487b3f128537ef49d40a08a"
server_key_version: "encryptionkey@0"
I20260809 04:46:56.007118 27999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "64231ecf75d0401eb83736280912fafd"
format_stamp: "Formatted at 2026-08-09 04:46:56 on dist-test-slave-1jjb"
server_key: "9ee91f2db204f69c8470e2f2e43a29d5"
server_key_iv: "c4c3bbc36487b3f128537ef49d40a08a"
server_key_version: "encryptionkey@0"
I20260809 04:46:56.011831 27999 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260809 04:46:56.014976 28014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:56.016234 27999 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:46:56.016392 27999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "64231ecf75d0401eb83736280912fafd"
format_stamp: "Formatted at 2026-08-09 04:46:56 on dist-test-slave-1jjb"
server_key: "9ee91f2db204f69c8470e2f2e43a29d5"
server_key_iv: "c4c3bbc36487b3f128537ef49d40a08a"
server_key_version: "encryptionkey@0"
I20260809 04:46:56.016548 27999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:46:56.034925 27999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:46:56.038450 27999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:46:56.038646 27999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:46:56.055668 27999 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:42407
I20260809 04:46:56.055697 28074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:42407 every 8 connection(s)
I20260809 04:46:56.056818 27999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:46:56.058810 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 27999
I20260809 04:46:56.059049 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:46:56.059666 13224 external_mini_cluster.cc:1428] Setting key b4c33507982edcb6ae5ac8d8ce1003ff
I20260809 04:46:56.060261 28075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:46:56.074798 28075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64231ecf75d0401eb83736280912fafd: Bootstrap starting.
Aug 09 04:46:56 dist-test-slave-1jjb krb5kdc[27984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:46:56.077600 28075 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64231ecf75d0401eb83736280912fafd: Neither blocks nor log segments found. Creating new log.
I20260809 04:46:56.078547 28075 log.cc:824] T 00000000000000000000000000000000 P 64231ecf75d0401eb83736280912fafd: Log is configured to *not* fsync() on all Append() calls
I20260809 04:46:56.081141 28075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64231ecf75d0401eb83736280912fafd: No bootstrap required, opened a new log
I20260809 04:46:56.082787 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:56.061462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56094 (local address 127.12.234.62:42407)
0809 04:46:56.062133 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:56.062141 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:56.062176 (+ 35us) server_negotiation.cc:407] Connection header received
0809 04:46:56.064446 (+ 2270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:56.064467 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:56.064792 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:56.065071 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:56.067050 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:56.068524 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:56.070563 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:56.071316 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:56.071463 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:56.076161 (+ 4698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:56.076168 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:56.076180 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:56.076219 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:56.079060 (+ 2841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:56.079807 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:56.079827 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:56.079833 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:56.079991 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:56.080428 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:56.080435 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:56.080442 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:56.080924 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:56.081195 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:56.081603 (+ 408us) server_negotiation.cc:299] Negotiation successful
0809 04:46:56.081953 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":320,"threads_started":1}
I20260809 04:46:56.086335 13224 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:46:56.088552 28075 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64231ecf75d0401eb83736280912fafd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64231ecf75d0401eb83736280912fafd" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42407 } }
I20260809 04:46:56.088905 28075 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64231ecf75d0401eb83736280912fafd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:46:56.089012 28075 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64231ecf75d0401eb83736280912fafd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64231ecf75d0401eb83736280912fafd, State: Initialized, Role: FOLLOWER
I20260809 04:46:56.089597 28075 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64231ecf75d0401eb83736280912fafd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64231ecf75d0401eb83736280912fafd" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42407 } }
I20260809 04:46:56.089857 28075 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64231ecf75d0401eb83736280912fafd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:46:56.090000 28075 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64231ecf75d0401eb83736280912fafd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:46:56.090157 28075 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64231ecf75d0401eb83736280912fafd [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:46:56.092129 28075 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64231ecf75d0401eb83736280912fafd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64231ecf75d0401eb83736280912fafd" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42407 } }
I20260809 04:46:56.092567 28075 leader_election.cc:302] T 00000000000000000000000000000000 P 64231ecf75d0401eb83736280912fafd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64231ecf75d0401eb83736280912fafd; no voters:
I20260809 04:46:56.093107 28075 leader_election.cc:288] T 00000000000000000000000000000000 P 64231ecf75d0401eb83736280912fafd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:46:56.093261 28080 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64231ecf75d0401eb83736280912fafd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:46:56.093997 28080 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64231ecf75d0401eb83736280912fafd [term 1 LEADER]: Becoming Leader. State: Replica: 64231ecf75d0401eb83736280912fafd, State: Running, Role: LEADER
I20260809 04:46:56.094559 28080 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64231ecf75d0401eb83736280912fafd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64231ecf75d0401eb83736280912fafd" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42407 } }
I20260809 04:46:56.095052 28075 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64231ecf75d0401eb83736280912fafd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:46:56.097506 28081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64231ecf75d0401eb83736280912fafd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64231ecf75d0401eb83736280912fafd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64231ecf75d0401eb83736280912fafd" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42407 } } }
I20260809 04:46:56.097535 28082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64231ecf75d0401eb83736280912fafd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64231ecf75d0401eb83736280912fafd. Latest consensus state: current_term: 1 leader_uuid: "64231ecf75d0401eb83736280912fafd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64231ecf75d0401eb83736280912fafd" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42407 } } }
I20260809 04:46:56.097973 28082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64231ecf75d0401eb83736280912fafd [sys.catalog]: This master's current role is: LEADER
I20260809 04:46:56.097972 28081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64231ecf75d0401eb83736280912fafd [sys.catalog]: This master's current role is: LEADER
I20260809 04:46:56.099856 28087 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:46:56.104127 28087 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:46:56.112365 28087 catalog_manager.cc:1347] Generated new cluster ID: f73cf912894c4f229ccef4acec686d69
I20260809 04:46:56.112462 28087 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:46:56.138711 28087 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:46:56.140748 28087 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:46:56.150736 28087 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64231ecf75d0401eb83736280912fafd: Generated new TSK 0
I20260809 04:46:56.151435 28087 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:46:56.198995 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 27999
I20260809 04:46:56.200145 28070 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:46:56.290024 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 27999
2026-08-09T04:46:56Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1963 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:46:56 dist-test-slave-1jjb krb5kdc[28107](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:46:56 dist-test-slave-1jjb krb5kdc[28107](info): set up 2 sockets
Aug 09 04:46:56 dist-test-slave-1jjb krb5kdc[28107](info): commencing operation
krb5kdc: starting...
W20260809 04:46:57.389011 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:46:57 dist-test-slave-1jjb krb5kdc[28107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:46:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:46:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:46:57.615873 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:44829
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:44829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:46:57.922412 28121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:46:57.922665 28121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:46:57.922741 28121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:46:57.928921 28121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:46:57.929001 28121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:46:57.929062 28121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:46:57.929133 28121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:46:57.929179 28121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:46:57.938201 28121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:44829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:44829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:46:57.939232 28121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:46:57.940421 28121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:46:57.951175 28130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:57.951189 28128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:57.951440 28132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:57.952028 28121 server_base.cc:1034] running on GCE node
I20260809 04:46:57.952757 28121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:46:57.953998 28121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:46:57.955205 28121 hybrid_clock.cc:648] HybridClock initialized: now 1786250817955171 us; error 25 us; skew 500 ppm
I20260809 04:46:57.955539 28121 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:46:57 dist-test-slave-1jjb krb5kdc[28107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250817, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:46:57.962628 28121 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:46:57.963976 28121 webserver.cc:460] Webserver started at http://127.12.234.62:36611/ using document root <none> and password file <none>
I20260809 04:46:57.964592 28121 fs_manager.cc:359] Metadata directory not provided
I20260809 04:46:57.964715 28121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:46:57.964993 28121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:46:57.967545 28121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "03dfd2bb88764bb9b513bf65e731fbca"
format_stamp: "Formatted at 2026-08-09 04:46:57 on dist-test-slave-1jjb"
server_key: "8ac237defb469ddc13cc830fb16d9b56"
server_key_iv: "57fc15cfc39ce527d14eb7a3450eca20"
server_key_version: "encryptionkey@0"
I20260809 04:46:57.968250 28121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "03dfd2bb88764bb9b513bf65e731fbca"
format_stamp: "Formatted at 2026-08-09 04:46:57 on dist-test-slave-1jjb"
server_key: "8ac237defb469ddc13cc830fb16d9b56"
server_key_iv: "57fc15cfc39ce527d14eb7a3450eca20"
server_key_version: "encryptionkey@0"
I20260809 04:46:57.973101 28121 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.002s
I20260809 04:46:57.976126 28140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:57.977425 28121 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:46:57.977598 28121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "03dfd2bb88764bb9b513bf65e731fbca"
format_stamp: "Formatted at 2026-08-09 04:46:57 on dist-test-slave-1jjb"
server_key: "8ac237defb469ddc13cc830fb16d9b56"
server_key_iv: "57fc15cfc39ce527d14eb7a3450eca20"
server_key_version: "encryptionkey@0"
I20260809 04:46:57.977767 28121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:46:57.998382 28121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:46:58.001994 28121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:46:58.002269 28121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:46:58.020153 28121 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:44829
I20260809 04:46:58.020160 28195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:44829 every 8 connection(s)
I20260809 04:46:58.021363 28121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:46:58.024845 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 28121
I20260809 04:46:58.024963 28196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:46:58.025063 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:46:58.025856 13224 external_mini_cluster.cc:1428] Setting key a0e81df4d16cb7f639e6a9259b47b17c
I20260809 04:46:58.038657 28196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca: Bootstrap starting.
Aug 09 04:46:58 dist-test-slave-1jjb krb5kdc[28107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:46:58.041710 28196 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca: Neither blocks nor log segments found. Creating new log.
I20260809 04:46:58.042923 28196 log.cc:824] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca: Log is configured to *not* fsync() on all Append() calls
I20260809 04:46:58.045866 28196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca: No bootstrap required, opened a new log
I20260809 04:46:58.049039 28200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:58.027824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53630 (local address 127.12.234.62:44829)
0809 04:46:58.028660 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:58.028671 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:58.028708 (+ 37us) server_negotiation.cc:407] Connection header received
0809 04:46:58.031488 (+ 2780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:58.031515 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:58.031971 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:58.032409 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:58.033542 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:58.035532 (+ 1990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:58.036857 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:58.037753 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:58.037920 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:58.041746 (+ 3826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:58.041755 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:58.041772 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:58.041808 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:58.045351 (+ 3543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:58.045914 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:58.045922 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:58.045928 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:58.046100 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:58.046427 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:58.046431 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:58.046433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:58.047019 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:58.047289 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:58.047525 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 04:46:58.047904 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":419,"threads_started":1}
I20260809 04:46:58.053702 28196 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03dfd2bb88764bb9b513bf65e731fbca" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44829 } }
I20260809 04:46:58.053978 28196 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:46:58.054044 28196 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03dfd2bb88764bb9b513bf65e731fbca, State: Initialized, Role: FOLLOWER
I20260809 04:46:58.054458 28196 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03dfd2bb88764bb9b513bf65e731fbca" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44829 } }
I20260809 04:46:58.054636 28196 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:46:58.054720 28196 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:46:58.054832 28196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:46:58.056705 28196 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03dfd2bb88764bb9b513bf65e731fbca" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44829 } }
I20260809 04:46:58.057150 28196 leader_election.cc:302] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03dfd2bb88764bb9b513bf65e731fbca; no voters:
I20260809 04:46:58.057787 28196 leader_election.cc:288] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:46:58.057860 28202 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:46:58.058594 28202 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [term 1 LEADER]: Becoming Leader. State: Replica: 03dfd2bb88764bb9b513bf65e731fbca, State: Running, Role: LEADER
I20260809 04:46:58.059118 28202 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03dfd2bb88764bb9b513bf65e731fbca" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44829 } }
I20260809 04:46:58.059692 28196 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:46:58.061657 28203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03dfd2bb88764bb9b513bf65e731fbca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03dfd2bb88764bb9b513bf65e731fbca" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44829 } } }
I20260809 04:46:58.061964 28203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [sys.catalog]: This master's current role is: LEADER
I20260809 04:46:58.062587 28204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03dfd2bb88764bb9b513bf65e731fbca. Latest consensus state: current_term: 1 leader_uuid: "03dfd2bb88764bb9b513bf65e731fbca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03dfd2bb88764bb9b513bf65e731fbca" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44829 } } }
I20260809 04:46:58.062804 28207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:46:58.062937 28204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [sys.catalog]: This master's current role is: LEADER
I20260809 04:46:58.068660 28207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:46:58.077625 28207 catalog_manager.cc:1347] Generated new cluster ID: 76c454d5ebc84f08a710b36bf9389929
I20260809 04:46:58.077723 28207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:46:58.119395 28207 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:46:58.121277 28207 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:46:58.132934 28207 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca: Generated new TSK 0
I20260809 04:46:58.133780 28207 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
I20260809 04:46:58.190927 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:0
--local_ip_for_outbound_sockets=127.12.234.1
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44829
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:46:58.497393 28227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:46:58.497622 28227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:46:58.497704 28227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:46:58.504230 28227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:46:58.504312 28227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:46:58.504415 28227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:46:58.513243 28227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:46:58.514202 28227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:46:58.515275 28227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:46:58.527282 28233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:58.527318 28232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:58.527344 28235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:58.528000 28227 server_base.cc:1034] running on GCE node
I20260809 04:46:58.528527 28227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:46:58.529359 28227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:46:58.530622 28227 hybrid_clock.cc:648] HybridClock initialized: now 1786250818530564 us; error 58 us; skew 500 ppm
I20260809 04:46:58.530980 28227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:46:58 dist-test-slave-1jjb krb5kdc[28107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250818, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:46:58.533979 28227 init.cc:377] Logged in from keytab as kudu/127.12.234.1@KRBTEST.COM (short username kudu)
I20260809 04:46:58.535282 28227 webserver.cc:460] Webserver started at http://127.12.234.1:40639/ using document root <none> and password file <none>
I20260809 04:46:58.535934 28227 fs_manager.cc:359] Metadata directory not provided
I20260809 04:46:58.536172 28227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:46:58.536527 28227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:46:58.539479 28227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/data/instance:
uuid: "a47af5c80dac43b19d033adba1a62d4b"
format_stamp: "Formatted at 2026-08-09 04:46:58 on dist-test-slave-1jjb"
server_key: "8f92676e2861bac6ef2bd56ea15ca845"
server_key_iv: "bc129a89fdc895567f2f6c14cf942498"
server_key_version: "encryptionkey@0"
I20260809 04:46:58.540434 28227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance:
uuid: "a47af5c80dac43b19d033adba1a62d4b"
format_stamp: "Formatted at 2026-08-09 04:46:58 on dist-test-slave-1jjb"
server_key: "8f92676e2861bac6ef2bd56ea15ca845"
server_key_iv: "bc129a89fdc895567f2f6c14cf942498"
server_key_version: "encryptionkey@0"
I20260809 04:46:58.545416 28227 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 04:46:58.549074 28243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:58.550390 28227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 04:46:58.550575 28227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "a47af5c80dac43b19d033adba1a62d4b"
format_stamp: "Formatted at 2026-08-09 04:46:58 on dist-test-slave-1jjb"
server_key: "8f92676e2861bac6ef2bd56ea15ca845"
server_key_iv: "bc129a89fdc895567f2f6c14cf942498"
server_key_version: "encryptionkey@0"
I20260809 04:46:58.550786 28227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:46:58.568380 28227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:46:58.572352 28227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:46:58.572654 28227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:46:58.573516 28227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:46:58.574692 28227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:46:58.574786 28227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:58.574915 28227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:46:58.574997 28227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:58.606272 28227 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:44033
I20260809 04:46:58.606345 28365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:44033 every 8 connection(s)
I20260809 04:46:58.607365 28227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:46:58.611495 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 28227
I20260809 04:46:58.611703 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance
I20260809 04:46:58.612574 13224 external_mini_cluster.cc:1428] Setting key a5b84d44024b90ecc501ff448b76826f
Aug 09 04:46:58 dist-test-slave-1jjb krb5kdc[28107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250818, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:46:58.628908 28369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:58.609658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:32959 (local address 127.12.234.62:44829)
0809 04:46:58.610114 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:58.610119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:58.610998 (+ 879us) server_negotiation.cc:407] Connection header received
0809 04:46:58.613561 (+ 2563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:58.613566 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:58.613652 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:58.613772 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:58.615411 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:58.616156 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:58.618492 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:58.618872 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:58.618965 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:58.623060 (+ 4095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:58.623066 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:58.623069 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:58.623102 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:58.625761 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:58.626514 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:58.626523 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:58.626528 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:58.626687 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:58.627146 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:58.627151 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:58.627153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:58.627433 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:58.627613 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:58.628272 (+ 659us) server_negotiation.cc:299] Negotiation successful
0809 04:46:58.628502 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":233,"threads_started":1}
I20260809 04:46:58.629881 28371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:58.610203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.1:32959)
0809 04:46:58.610843 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:58.610886 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:58.613298 (+ 2412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:58.614015 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:58.614022 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:58.614537 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:46:58.615223 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:58.615247 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:58.616321 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:58.616328 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:58.618331 (+ 2003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:58.618348 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:58.619099 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:58.619108 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:58.619416 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:58.620142 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:46:58.620182 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:46:58.622826 (+ 2644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:46:58.625929 (+ 3103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:58.625938 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:58.625951 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:58.626378 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:58.626803 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:58.626808 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:58.626814 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:58.627004 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:58.627583 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:46:58.627591 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:46:58.627888 (+ 297us) client_negotiation.cc:769] Sending connection context
0809 04:46:58.628195 (+ 307us) client_negotiation.cc:240] Negotiation successful
0809 04:46:58.628656 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":250,"threads_started":1}
I20260809 04:46:58.631649 28366 heartbeater.cc:347] Connected to a master server at 127.12.234.62:44829
I20260809 04:46:58.631943 28366 heartbeater.cc:464] Registering TS with master...
I20260809 04:46:58.632831 28366 heartbeater.cc:511] Master 127.12.234.62:44829 requested a full tablet report, sending...
I20260809 04:46:58.634958 28159 ts_manager.cc:194] Registered new tserver with Master: a47af5c80dac43b19d033adba1a62d4b (127.12.234.1:44033)
I20260809 04:46:58.636515 28159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.1@KRBTEST.COM'} at 127.12.234.1:32959
WARNING: no policy specified for kudu/127.12.234.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
I20260809 04:46:58.671213 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:0
--local_ip_for_outbound_sockets=127.12.234.2
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44829
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:46:58.975683 28374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:46:58.975915 28374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:46:58.976001 28374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:46:58.982784 28374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:46:58.982874 28374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:46:58.982975 28374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:46:58.991916 28374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:46:58.992991 28374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:46:58.994107 28374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:46:59.005834 28380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:59.005869 28384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:59.006111 28381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:59.007027 28374 server_base.cc:1034] running on GCE node
I20260809 04:46:59.007673 28374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:46:59.008684 28374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:46:59.009913 28374 hybrid_clock.cc:648] HybridClock initialized: now 1786250819009869 us; error 53 us; skew 500 ppm
I20260809 04:46:59.010218 28374 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:46:59 dist-test-slave-1jjb krb5kdc[28107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250819, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:46:59.013223 28374 init.cc:377] Logged in from keytab as kudu/127.12.234.2@KRBTEST.COM (short username kudu)
I20260809 04:46:59.014499 28374 webserver.cc:460] Webserver started at http://127.12.234.2:39859/ using document root <none> and password file <none>
I20260809 04:46:59.015115 28374 fs_manager.cc:359] Metadata directory not provided
I20260809 04:46:59.015259 28374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:46:59.015553 28374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:46:59.018134 28374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/data/instance:
uuid: "2a69dc7a46a140f49df77a742cc1c828"
format_stamp: "Formatted at 2026-08-09 04:46:59 on dist-test-slave-1jjb"
server_key: "8fe6ece157b8fe538c15637314298f89"
server_key_iv: "f0e39962459b4d96cca01c76fb180c80"
server_key_version: "encryptionkey@0"
I20260809 04:46:59.018848 28374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance:
uuid: "2a69dc7a46a140f49df77a742cc1c828"
format_stamp: "Formatted at 2026-08-09 04:46:59 on dist-test-slave-1jjb"
server_key: "8fe6ece157b8fe538c15637314298f89"
server_key_iv: "f0e39962459b4d96cca01c76fb180c80"
server_key_version: "encryptionkey@0"
I20260809 04:46:59.023643 28374 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260809 04:46:59.027096 28391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:59.028388 28374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260809 04:46:59.028532 28374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "2a69dc7a46a140f49df77a742cc1c828"
format_stamp: "Formatted at 2026-08-09 04:46:59 on dist-test-slave-1jjb"
server_key: "8fe6ece157b8fe538c15637314298f89"
server_key_iv: "f0e39962459b4d96cca01c76fb180c80"
server_key_version: "encryptionkey@0"
I20260809 04:46:59.028694 28374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:46:59.049801 28374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:46:59.053313 28374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:46:59.053580 28374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:46:59.054469 28374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:46:59.055713 28374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:46:59.055778 28374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:59.055919 28374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:46:59.055984 28374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:59.087932 28374 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:45601
I20260809 04:46:59.088009 28519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:45601 every 8 connection(s)
I20260809 04:46:59.089108 28374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:46:59.091702 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 28374
I20260809 04:46:59.091892 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance
I20260809 04:46:59.092403 13224 external_mini_cluster.cc:1428] Setting key a5ccc6cb7d92d479a63f49593e03a5a3
Aug 09 04:46:59 dist-test-slave-1jjb krb5kdc[28107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250819, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:46:59.109558 28369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:59.091563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:37383 (local address 127.12.234.62:44829)
0809 04:46:59.091792 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:59.091801 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:59.093154 (+ 1353us) server_negotiation.cc:407] Connection header received
0809 04:46:59.094904 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:59.094910 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:59.094999 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:59.095105 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:59.096399 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.097599 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.099022 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.099690 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.099825 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:59.103196 (+ 3371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:59.103204 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:59.103211 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:59.103254 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:59.106390 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:59.107259 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:59.107262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:59.107265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:59.107400 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:59.107833 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:59.107835 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:59.107837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:59.108215 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:59.108330 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:59.109161 (+ 831us) server_negotiation.cc:299] Negotiation successful
0809 04:46:59.109327 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 04:46:59.111040 28522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:59.092198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.2:37383)
0809 04:46:59.093013 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:59.093055 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:59.094660 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:59.095269 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:59.095276 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:59.095696 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:46:59.096246 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:59.096264 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.097750 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.097754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:59.098882 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:59.098893 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.099871 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.099875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:59.100140 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:59.100861 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:46:59.100882 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:46:59.102995 (+ 2113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:46:59.106523 (+ 3528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:59.106533 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:59.106551 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:59.107140 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:59.107503 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:59.107510 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:59.107515 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:59.107728 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:59.108339 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:46:59.108349 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:46:59.108762 (+ 413us) client_negotiation.cc:769] Sending connection context
0809 04:46:59.109072 (+ 310us) client_negotiation.cc:240] Negotiation successful
0809 04:46:59.109629 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":332,"threads_started":1}
I20260809 04:46:59.112397 28520 heartbeater.cc:347] Connected to a master server at 127.12.234.62:44829
I20260809 04:46:59.112679 28520 heartbeater.cc:464] Registering TS with master...
I20260809 04:46:59.113374 28520 heartbeater.cc:511] Master 127.12.234.62:44829 requested a full tablet report, sending...
I20260809 04:46:59.114781 28159 ts_manager.cc:194] Registered new tserver with Master: 2a69dc7a46a140f49df77a742cc1c828 (127.12.234.2:45601)
I20260809 04:46:59.115546 28159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.2@KRBTEST.COM'} at 127.12.234.2:37383
WARNING: no policy specified for kudu/127.12.234.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
I20260809 04:46:59.152369 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:0
--local_ip_for_outbound_sockets=127.12.234.3
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44829
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:46:59.455704 28526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:46:59.455976 28526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:46:59.456175 28526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:46:59.462831 28526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:46:59.462960 28526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:46:59.463119 28526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:46:59.472707 28526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:46:59.473886 28526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:46:59.475565 28526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:46:59.490211 28532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:59.490453 28533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:46:59.490465 28535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:46:59.491447 28526 server_base.cc:1034] running on GCE node
I20260809 04:46:59.492063 28526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:46:59.492944 28526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:46:59.494136 28526 hybrid_clock.cc:648] HybridClock initialized: now 1786250819494090 us; error 39 us; skew 500 ppm
I20260809 04:46:59.494463 28526 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:46:59 dist-test-slave-1jjb krb5kdc[28107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250819, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:46:59.497486 28526 init.cc:377] Logged in from keytab as kudu/127.12.234.3@KRBTEST.COM (short username kudu)
I20260809 04:46:59.498772 28526 webserver.cc:460] Webserver started at http://127.12.234.3:44661/ using document root <none> and password file <none>
I20260809 04:46:59.499353 28526 fs_manager.cc:359] Metadata directory not provided
I20260809 04:46:59.499470 28526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:46:59.499750 28526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:46:59.502602 28526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/data/instance:
uuid: "ca00b8cfda2742ff9e79c74f2c62a843"
format_stamp: "Formatted at 2026-08-09 04:46:59 on dist-test-slave-1jjb"
server_key: "38339b9d93170199d348fe64d6b41f5c"
server_key_iv: "6424c17a5cb5be714b43c3870b615102"
server_key_version: "encryptionkey@0"
I20260809 04:46:59.503309 28526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance:
uuid: "ca00b8cfda2742ff9e79c74f2c62a843"
format_stamp: "Formatted at 2026-08-09 04:46:59 on dist-test-slave-1jjb"
server_key: "38339b9d93170199d348fe64d6b41f5c"
server_key_iv: "6424c17a5cb5be714b43c3870b615102"
server_key_version: "encryptionkey@0"
I20260809 04:46:59.507902 28526 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260809 04:46:59.511348 28542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:59.512645 28526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:46:59.512821 28526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "ca00b8cfda2742ff9e79c74f2c62a843"
format_stamp: "Formatted at 2026-08-09 04:46:59 on dist-test-slave-1jjb"
server_key: "38339b9d93170199d348fe64d6b41f5c"
server_key_iv: "6424c17a5cb5be714b43c3870b615102"
server_key_version: "encryptionkey@0"
I20260809 04:46:59.513036 28526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:46:59.524062 28526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:46:59.527452 28526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:46:59.527695 28526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:46:59.528501 28526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:46:59.529649 28526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:46:59.529718 28526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:59.529884 28526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:46:59.529934 28526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:46:59.559638 28526 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:45063
I20260809 04:46:59.559710 28671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:45063 every 8 connection(s)
I20260809 04:46:59.560868 28526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:46:59.562176 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 28526
I20260809 04:46:59.562335 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance
I20260809 04:46:59.562760 13224 external_mini_cluster.cc:1428] Setting key 1219b1b7b93d2bb3f962d44efc9e3576
Aug 09 04:46:59 dist-test-slave-1jjb krb5kdc[28107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250819, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:46:59.579311 28369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:59.563293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:54261 (local address 127.12.234.62:44829)
0809 04:46:59.563484 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:59.563505 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:59.564714 (+ 1209us) server_negotiation.cc:407] Connection header received
0809 04:46:59.566856 (+ 2142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:59.566859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:59.566918 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:59.566989 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:59.568300 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.569026 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.570595 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.571018 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.571123 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:59.574073 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:59.574077 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:59.574080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:59.574109 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:59.576875 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:59.577670 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:59.577677 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:59.577680 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:59.577793 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:59.578153 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:59.578156 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:59.578157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:59.578377 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:59.578471 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:59.578956 (+ 485us) server_negotiation.cc:299] Negotiation successful
0809 04:46:59.579089 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 04:46:59.580260 28674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:59.563788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.3:54261)
0809 04:46:59.564568 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:59.564614 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:59.566642 (+ 2028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:59.567149 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:59.567155 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:59.567606 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:46:59.568136 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:59.568165 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.569152 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.569155 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:59.570452 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:59.570470 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.571202 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.571207 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:59.571380 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:59.571929 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:46:59.571949 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:46:59.573898 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:46:59.577002 (+ 3104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:59.577009 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:59.577022 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:59.577548 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:59.577882 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:46:59.577885 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:46:59.577887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:46:59.578067 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:46:59.578469 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:46:59.578473 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:46:59.578700 (+ 227us) client_negotiation.cc:769] Sending connection context
0809 04:46:59.578877 (+ 177us) client_negotiation.cc:240] Negotiation successful
0809 04:46:59.579174 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":343,"threads_started":1}
I20260809 04:46:59.581604 28672 heartbeater.cc:347] Connected to a master server at 127.12.234.62:44829
I20260809 04:46:59.581844 28672 heartbeater.cc:464] Registering TS with master...
I20260809 04:46:59.582391 28672 heartbeater.cc:511] Master 127.12.234.62:44829 requested a full tablet report, sending...
I20260809 04:46:59.583653 28159 ts_manager.cc:194] Registered new tserver with Master: ca00b8cfda2742ff9e79c74f2c62a843 (127.12.234.3:45063)
I20260809 04:46:59.584468 28159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.3@KRBTEST.COM'} at 127.12.234.3:54261
I20260809 04:46:59.589329 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:46:59.605172 28369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:59.593780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53644 (local address 127.12.234.62:44829)
0809 04:46:59.593941 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:59.593949 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:59.594308 (+ 359us) server_negotiation.cc:407] Connection header received
0809 04:46:59.594497 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:59.594502 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:59.594568 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:59.594659 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:59.595886 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.596591 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.597772 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.598164 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.598279 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:59.600496 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:59.600501 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:59.600504 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:59.600530 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:59.602728 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:59.603306 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:59.603311 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:59.603315 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:59.603452 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:59.603817 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:59.603822 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:59.603827 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:59.604155 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:59.604395 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:59.604712 (+ 317us) server_negotiation.cc:299] Negotiation successful
0809 04:46:59.604903 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 04:46:59.608831 28159 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53644:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:46:59.615418 28159 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:46:59.639204 28366 heartbeater.cc:503] Master 127.12.234.62:44829 was elected leader, sending a full tablet report...
I20260809 04:46:59.649251 28689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:59.626682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:45063 (local address 127.0.0.1:45546)
0809 04:46:59.638635 (+ 11953us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:59.638671 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:59.638843 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:59.641249 (+ 2406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:59.641259 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:59.641300 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:46:59.641868 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:59.641884 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.644928 (+ 3044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.644932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:59.646932 (+ 2000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:59.646943 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.648384 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.648389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:59.648558 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:59.648606 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 04:46:59.648721 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 04:46:59.648884 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11755,"thread_start_us":14690,"threads_started":1}
I20260809 04:46:59.649412 28687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:59.626957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45546 (local address 127.12.234.3:45063)
0809 04:46:59.638121 (+ 11164us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:59.638132 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:59.639593 (+ 1461us) server_negotiation.cc:407] Connection header received
0809 04:46:59.639715 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:59.639722 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:59.639965 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:59.640876 (+ 911us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:46:59.643188 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.644749 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.647076 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.647926 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.648031 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:59.648250 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:59.648890 (+ 640us) server_negotiation.cc:299] Negotiation successful
0809 04:46:59.649077 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10961,"thread_start_us":10790,"threads_started":1}
I20260809 04:46:59.650326 28685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:59.626730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:45601 (local address 127.0.0.1:45598)
0809 04:46:59.641098 (+ 14368us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:59.641121 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:59.641273 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:59.643418 (+ 2145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:59.643422 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:59.643443 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:46:59.643825 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:59.643837 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.645776 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.645783 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:59.647661 (+ 1878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:59.647675 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.649548 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.649553 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:59.649711 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:59.649741 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 04:46:59.649824 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 04:46:59.649944 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14206,"mutex_wait_us":15,"thread_start_us":16170,"threads_started":1}
I20260809 04:46:59.650520 28688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:59.627235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45598 (local address 127.12.234.2:45601)
0809 04:46:59.642870 (+ 15635us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:59.642881 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:59.642911 (+ 30us) server_negotiation.cc:407] Connection header received
0809 04:46:59.643010 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:59.643017 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:59.643223 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:59.644005 (+ 782us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:46:59.644273 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.645645 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.647818 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.649073 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.649213 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:59.649425 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:59.650041 (+ 616us) server_negotiation.cc:299] Negotiation successful
0809 04:46:59.650201 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15441,"thread_start_us":15703,"threads_started":1}
I20260809 04:46:59.651221 28686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:59.627263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:44033 (local address 127.0.0.1:59806)
0809 04:46:59.643187 (+ 15924us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:59.643204 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:59.643390 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:59.644204 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:59.644208 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:59.644232 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:46:59.644525 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:59.644533 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.646416 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.646421 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:59.648334 (+ 1913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:59.648346 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.650417 (+ 2071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.650426 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:59.650604 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:59.650643 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 04:46:59.650727 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 04:46:59.650861 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15798,"mutex_wait_us":21,"thread_start_us":11044,"threads_started":1}
I20260809 04:46:59.651523 28690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:59.627854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59806 (local address 127.12.234.1:44033)
0809 04:46:59.642892 (+ 15038us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:59.642902 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:59.643293 (+ 391us) server_negotiation.cc:407] Connection header received
0809 04:46:59.643572 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:59.643580 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:59.643773 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:59.643919 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:46:59.644676 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.646282 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.648511 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.649908 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.650064 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:59.650268 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:59.651034 (+ 766us) server_negotiation.cc:299] Negotiation successful
0809 04:46:59.651213 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14868,"thread_start_us":14712,"threads_started":1}
I20260809 04:46:59.653699 28448 tablet_service.cc:1467] Processing CreateTablet for tablet a0ad134e0bab43d6b9a63b04e05e9b97 (DEFAULT_TABLE table=test-table [id=c4ce41aa9e894098af007fbc29393da8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:46:59.653613 28600 tablet_service.cc:1467] Processing CreateTablet for tablet a0ad134e0bab43d6b9a63b04e05e9b97 (DEFAULT_TABLE table=test-table [id=c4ce41aa9e894098af007fbc29393da8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:46:59.654846 28448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0ad134e0bab43d6b9a63b04e05e9b97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:46:59.655109 28600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0ad134e0bab43d6b9a63b04e05e9b97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:46:59.654886 28295 tablet_service.cc:1467] Processing CreateTablet for tablet a0ad134e0bab43d6b9a63b04e05e9b97 (DEFAULT_TABLE table=test-table [id=c4ce41aa9e894098af007fbc29393da8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:46:59.656212 28295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0ad134e0bab43d6b9a63b04e05e9b97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:46:59.665618 28691 tablet_bootstrap.cc:492] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828: Bootstrap starting.
I20260809 04:46:59.668417 28692 tablet_bootstrap.cc:492] T a0ad134e0bab43d6b9a63b04e05e9b97 P ca00b8cfda2742ff9e79c74f2c62a843: Bootstrap starting.
I20260809 04:46:59.668754 28691 tablet_bootstrap.cc:654] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828: Neither blocks nor log segments found. Creating new log.
I20260809 04:46:59.670225 28691 log.cc:824] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828: Log is configured to *not* fsync() on all Append() calls
I20260809 04:46:59.672071 28692 tablet_bootstrap.cc:654] T a0ad134e0bab43d6b9a63b04e05e9b97 P ca00b8cfda2742ff9e79c74f2c62a843: Neither blocks nor log segments found. Creating new log.
I20260809 04:46:59.673265 28692 log.cc:824] T a0ad134e0bab43d6b9a63b04e05e9b97 P ca00b8cfda2742ff9e79c74f2c62a843: Log is configured to *not* fsync() on all Append() calls
I20260809 04:46:59.673368 28691 tablet_bootstrap.cc:492] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828: No bootstrap required, opened a new log
I20260809 04:46:59.673597 28691 ts_tablet_manager.cc:1397] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 04:46:59.674002 28693 tablet_bootstrap.cc:492] T a0ad134e0bab43d6b9a63b04e05e9b97 P a47af5c80dac43b19d033adba1a62d4b: Bootstrap starting.
I20260809 04:46:59.675549 28692 tablet_bootstrap.cc:492] T a0ad134e0bab43d6b9a63b04e05e9b97 P ca00b8cfda2742ff9e79c74f2c62a843: No bootstrap required, opened a new log
I20260809 04:46:59.675779 28692 ts_tablet_manager.cc:1397] T a0ad134e0bab43d6b9a63b04e05e9b97 P ca00b8cfda2742ff9e79c74f2c62a843: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260809 04:46:59.676966 28693 tablet_bootstrap.cc:654] T a0ad134e0bab43d6b9a63b04e05e9b97 P a47af5c80dac43b19d033adba1a62d4b: Neither blocks nor log segments found. Creating new log.
I20260809 04:46:59.677969 28693 log.cc:824] T a0ad134e0bab43d6b9a63b04e05e9b97 P a47af5c80dac43b19d033adba1a62d4b: Log is configured to *not* fsync() on all Append() calls
I20260809 04:46:59.682798 28693 tablet_bootstrap.cc:492] T a0ad134e0bab43d6b9a63b04e05e9b97 P a47af5c80dac43b19d033adba1a62d4b: No bootstrap required, opened a new log
I20260809 04:46:59.683080 28693 ts_tablet_manager.cc:1397] T a0ad134e0bab43d6b9a63b04e05e9b97 P a47af5c80dac43b19d033adba1a62d4b: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260809 04:46:59.683759 28691 raft_consensus.cc:348] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 } } peers { permanent_uuid: "2a69dc7a46a140f49df77a742cc1c828" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45601 } } peers { permanent_uuid: "a47af5c80dac43b19d033adba1a62d4b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 44033 } }
I20260809 04:46:59.683775 28692 raft_consensus.cc:348] T a0ad134e0bab43d6b9a63b04e05e9b97 P ca00b8cfda2742ff9e79c74f2c62a843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 } } peers { permanent_uuid: "2a69dc7a46a140f49df77a742cc1c828" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45601 } } peers { permanent_uuid: "a47af5c80dac43b19d033adba1a62d4b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 44033 } }
I20260809 04:46:59.684149 28691 raft_consensus.cc:374] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:46:59.684232 28692 raft_consensus.cc:374] T a0ad134e0bab43d6b9a63b04e05e9b97 P ca00b8cfda2742ff9e79c74f2c62a843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:46:59.684267 28691 raft_consensus.cc:729] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a69dc7a46a140f49df77a742cc1c828, State: Initialized, Role: FOLLOWER
I20260809 04:46:59.684333 28692 raft_consensus.cc:729] T a0ad134e0bab43d6b9a63b04e05e9b97 P ca00b8cfda2742ff9e79c74f2c62a843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca00b8cfda2742ff9e79c74f2c62a843, State: Initialized, Role: FOLLOWER
I20260809 04:46:59.684827 28691 consensus_queue.cc:260] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 } } peers { permanent_uuid: "2a69dc7a46a140f49df77a742cc1c828" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45601 } } peers { permanent_uuid: "a47af5c80dac43b19d033adba1a62d4b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 44033 } }
I20260809 04:46:59.684950 28692 consensus_queue.cc:260] T a0ad134e0bab43d6b9a63b04e05e9b97 P ca00b8cfda2742ff9e79c74f2c62a843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 } } peers { permanent_uuid: "2a69dc7a46a140f49df77a742cc1c828" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45601 } } peers { permanent_uuid: "a47af5c80dac43b19d033adba1a62d4b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 44033 } }
I20260809 04:46:59.685859 28520 heartbeater.cc:503] Master 127.12.234.62:44829 was elected leader, sending a full tablet report...
I20260809 04:46:59.686445 28691 ts_tablet_manager.cc:1428] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260809 04:46:59.687352 28672 heartbeater.cc:503] Master 127.12.234.62:44829 was elected leader, sending a full tablet report...
I20260809 04:46:59.687933 28692 ts_tablet_manager.cc:1428] T a0ad134e0bab43d6b9a63b04e05e9b97 P ca00b8cfda2742ff9e79c74f2c62a843: Time spent starting tablet: real 0.012s user 0.012s sys 0.001s
I20260809 04:46:59.693027 28693 raft_consensus.cc:348] T a0ad134e0bab43d6b9a63b04e05e9b97 P a47af5c80dac43b19d033adba1a62d4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 } } peers { permanent_uuid: "2a69dc7a46a140f49df77a742cc1c828" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45601 } } peers { permanent_uuid: "a47af5c80dac43b19d033adba1a62d4b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 44033 } }
I20260809 04:46:59.693287 28693 raft_consensus.cc:374] T a0ad134e0bab43d6b9a63b04e05e9b97 P a47af5c80dac43b19d033adba1a62d4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:46:59.693367 28693 raft_consensus.cc:729] T a0ad134e0bab43d6b9a63b04e05e9b97 P a47af5c80dac43b19d033adba1a62d4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a47af5c80dac43b19d033adba1a62d4b, State: Initialized, Role: FOLLOWER
I20260809 04:46:59.693814 28693 consensus_queue.cc:260] T a0ad134e0bab43d6b9a63b04e05e9b97 P a47af5c80dac43b19d033adba1a62d4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 } } peers { permanent_uuid: "2a69dc7a46a140f49df77a742cc1c828" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45601 } } peers { permanent_uuid: "a47af5c80dac43b19d033adba1a62d4b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 44033 } }
I20260809 04:46:59.695232 28693 ts_tablet_manager.cc:1428] T a0ad134e0bab43d6b9a63b04e05e9b97 P a47af5c80dac43b19d033adba1a62d4b: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260809 04:46:59.812436 28698 raft_consensus.cc:482] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 04:46:59.812435 28673 tablet.cc:2369] T a0ad134e0bab43d6b9a63b04e05e9b97 P ca00b8cfda2742ff9e79c74f2c62a843: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:46:59.812757 28698 raft_consensus.cc:504] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 } } peers { permanent_uuid: "2a69dc7a46a140f49df77a742cc1c828" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45601 } } peers { permanent_uuid: "a47af5c80dac43b19d033adba1a62d4b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 44033 } }
I20260809 04:46:59.814497 28698 leader_election.cc:288] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca00b8cfda2742ff9e79c74f2c62a843 (127.12.234.3:45063), a47af5c80dac43b19d033adba1a62d4b (127.12.234.1:44033)
I20260809 04:46:59.821709 28690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:59.814923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:38921 (local address 127.12.234.1:44033)
0809 04:46:59.815240 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:59.815249 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:59.816015 (+ 766us) server_negotiation.cc:407] Connection header received
0809 04:46:59.816262 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:59.816270 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:59.816381 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:59.816449 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:46:59.817113 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.817926 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.819875 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.820744 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.820900 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:59.820993 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:59.821327 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 04:46:59.821461 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260809 04:46:59.821727 28687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:59.814949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:51277 (local address 127.12.234.3:45063)
0809 04:46:59.815210 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:59.815220 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:59.816090 (+ 870us) server_negotiation.cc:407] Connection header received
0809 04:46:59.816370 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:59.816375 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:59.816473 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:59.816567 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:46:59.817328 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.818109 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.819915 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.820770 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.820955 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:59.821043 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:59.821362 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 04:46:59.821484 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 04:46:59.821838 28701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:59.814815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:44033 (local address 127.12.234.2:38921)
0809 04:46:59.815898 (+ 1083us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:59.815913 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:59.816062 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:59.816504 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:59.816511 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:59.816543 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:46:59.816961 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:59.816973 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.818068 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.818075 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:59.819696 (+ 1621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:59.819708 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.820989 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.820996 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:59.821155 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:59.821178 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 04:46:59.821266 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 04:46:59.821404 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":953,"mutex_wait_us":32,"thread_start_us":644,"threads_started":1}
I20260809 04:46:59.822270 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:59.814841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:45063 (local address 127.12.234.2:51277)
0809 04:46:59.815939 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0809 04:46:59.815954 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:46:59.816203 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:46:59.816606 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:46:59.816617 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:46:59.816650 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:46:59.817093 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:59.817105 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.818252 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.818256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:59.819722 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:46:59.819734 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.821029 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.821037 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:46:59.821175 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:59.821197 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 04:46:59.821318 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 04:46:59.821436 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":995,"mutex_wait_us":18,"thread_start_us":695,"threads_started":1}
I20260809 04:46:59.822956 28319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ad134e0bab43d6b9a63b04e05e9b97" candidate_uuid: "2a69dc7a46a140f49df77a742cc1c828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a47af5c80dac43b19d033adba1a62d4b" is_pre_election: true
I20260809 04:46:59.823263 28621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ad134e0bab43d6b9a63b04e05e9b97" candidate_uuid: "2a69dc7a46a140f49df77a742cc1c828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" is_pre_election: true
I20260809 04:46:59.823459 28319 raft_consensus.cc:2454] T a0ad134e0bab43d6b9a63b04e05e9b97 P a47af5c80dac43b19d033adba1a62d4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a69dc7a46a140f49df77a742cc1c828 in term 0.
I20260809 04:46:59.823766 28621 raft_consensus.cc:2454] T a0ad134e0bab43d6b9a63b04e05e9b97 P ca00b8cfda2742ff9e79c74f2c62a843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a69dc7a46a140f49df77a742cc1c828 in term 0.
I20260809 04:46:59.824249 28393 leader_election.cc:302] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a69dc7a46a140f49df77a742cc1c828, a47af5c80dac43b19d033adba1a62d4b; no voters:
I20260809 04:46:59.824710 28698 raft_consensus.cc:2781] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:46:59.824879 28698 raft_consensus.cc:482] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:46:59.824980 28698 raft_consensus.cc:3037] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:46:59.827204 28698 raft_consensus.cc:504] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 } } peers { permanent_uuid: "2a69dc7a46a140f49df77a742cc1c828" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45601 } } peers { permanent_uuid: "a47af5c80dac43b19d033adba1a62d4b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 44033 } }
I20260809 04:46:59.827842 28698 leader_election.cc:288] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828 [CANDIDATE]: Term 1 election: Requested vote from peers ca00b8cfda2742ff9e79c74f2c62a843 (127.12.234.3:45063), a47af5c80dac43b19d033adba1a62d4b (127.12.234.1:44033)
I20260809 04:46:59.828285 28319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ad134e0bab43d6b9a63b04e05e9b97" candidate_uuid: "2a69dc7a46a140f49df77a742cc1c828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a47af5c80dac43b19d033adba1a62d4b"
I20260809 04:46:59.828281 28621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0ad134e0bab43d6b9a63b04e05e9b97" candidate_uuid: "2a69dc7a46a140f49df77a742cc1c828" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca00b8cfda2742ff9e79c74f2c62a843"
I20260809 04:46:59.828457 28621 raft_consensus.cc:3037] T a0ad134e0bab43d6b9a63b04e05e9b97 P ca00b8cfda2742ff9e79c74f2c62a843 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:46:59.828470 28319 raft_consensus.cc:3037] T a0ad134e0bab43d6b9a63b04e05e9b97 P a47af5c80dac43b19d033adba1a62d4b [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:46:59.830641 28621 raft_consensus.cc:2454] T a0ad134e0bab43d6b9a63b04e05e9b97 P ca00b8cfda2742ff9e79c74f2c62a843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a69dc7a46a140f49df77a742cc1c828 in term 1.
I20260809 04:46:59.830652 28319 raft_consensus.cc:2454] T a0ad134e0bab43d6b9a63b04e05e9b97 P a47af5c80dac43b19d033adba1a62d4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a69dc7a46a140f49df77a742cc1c828 in term 1.
I20260809 04:46:59.831023 28397 leader_election.cc:302] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a69dc7a46a140f49df77a742cc1c828, ca00b8cfda2742ff9e79c74f2c62a843; no voters:
I20260809 04:46:59.831333 28698 raft_consensus.cc:2781] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:46:59.831763 28698 raft_consensus.cc:686] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828 [term 1 LEADER]: Becoming Leader. State: Replica: 2a69dc7a46a140f49df77a742cc1c828, State: Running, Role: LEADER
I20260809 04:46:59.832257 28698 consensus_queue.cc:237] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 } } peers { permanent_uuid: "2a69dc7a46a140f49df77a742cc1c828" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45601 } } peers { permanent_uuid: "a47af5c80dac43b19d033adba1a62d4b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 44033 } }
I20260809 04:46:59.836737 28159 catalog_manager.cc:5495] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a69dc7a46a140f49df77a742cc1c828 (127.12.234.2). New cstate: current_term: 1 leader_uuid: "2a69dc7a46a140f49df77a742cc1c828" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a69dc7a46a140f49df77a742cc1c828" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a47af5c80dac43b19d033adba1a62d4b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 44033 } health_report { overall_health: UNKNOWN } } }
W20260809 04:46:59.840955 28521 tablet.cc:2369] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:46:59.859448 28368 tablet.cc:2369] T a0ad134e0bab43d6b9a63b04e05e9b97 P a47af5c80dac43b19d033adba1a62d4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:46:59.884193 28688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:59.878749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45604 (local address 127.12.234.2:45601)
0809 04:46:59.878919 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:59.878924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:59.879013 (+ 89us) server_negotiation.cc:407] Connection header received
0809 04:46:59.879227 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:59.879231 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:59.879304 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:59.879453 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:46:59.879799 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.880601 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.882181 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.882592 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.882715 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:59.883203 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:46:59.883630 (+ 427us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:46:59.883769 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:59.883844 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 04:46:59.883916 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:46:59.891137 28621 raft_consensus.cc:1261] T a0ad134e0bab43d6b9a63b04e05e9b97 P ca00b8cfda2742ff9e79c74f2c62a843 [term 1 FOLLOWER]: Refusing update from remote peer 2a69dc7a46a140f49df77a742cc1c828: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:46:59.891139 28319 raft_consensus.cc:1261] T a0ad134e0bab43d6b9a63b04e05e9b97 P a47af5c80dac43b19d033adba1a62d4b [term 1 FOLLOWER]: Refusing update from remote peer 2a69dc7a46a140f49df77a742cc1c828: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:46:59.892115 28703 consensus_queue.cc:1035] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a47af5c80dac43b19d033adba1a62d4b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 44033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:46:59.892452 28698 consensus_queue.cc:1035] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:46:59.907348 28708 mvcc.cc:205] Tried to move back new op lower bound from 7316483358264647680 to 7316483358036385792. Current Snapshot: MvccSnapshot[applied={T|T < 7316483358264647680}]
I20260809 04:46:59.916283 28159 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:46:59.916505 28159 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:46:59.919971 28159 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca: Sending DeleteTablet for 3 replicas of tablet a0ad134e0bab43d6b9a63b04e05e9b97
I20260809 04:46:59.921048 28600 tablet_service.cc:1514] Processing DeleteTablet for tablet a0ad134e0bab43d6b9a63b04e05e9b97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:46:59 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:45546
I20260809 04:46:59.921202 28448 tablet_service.cc:1514] Processing DeleteTablet for tablet a0ad134e0bab43d6b9a63b04e05e9b97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:46:59 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:45598
I20260809 04:46:59.921393 28295 tablet_service.cc:1514] Processing DeleteTablet for tablet a0ad134e0bab43d6b9a63b04e05e9b97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:46:59 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:59806
I20260809 04:46:59.922574 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 28227
I20260809 04:46:59.934175 28718 tablet_replica.cc:330] T a0ad134e0bab43d6b9a63b04e05e9b97 P ca00b8cfda2742ff9e79c74f2c62a843: stopping tablet replica
I20260809 04:46:59.934587 28718 raft_consensus.cc:2229] T a0ad134e0bab43d6b9a63b04e05e9b97 P ca00b8cfda2742ff9e79c74f2c62a843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:46:59.934937 28720 tablet_replica.cc:330] T a0ad134e0bab43d6b9a63b04e05e9b97 P a47af5c80dac43b19d033adba1a62d4b: stopping tablet replica
I20260809 04:46:59.935076 28718 raft_consensus.cc:2258] T a0ad134e0bab43d6b9a63b04e05e9b97 P ca00b8cfda2742ff9e79c74f2c62a843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:46:59.936115 28720 raft_consensus.cc:2229] T a0ad134e0bab43d6b9a63b04e05e9b97 P a47af5c80dac43b19d033adba1a62d4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:46:59.937206 28718 ts_tablet_manager.cc:1905] T a0ad134e0bab43d6b9a63b04e05e9b97 P ca00b8cfda2742ff9e79c74f2c62a843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:46:59.937264 28720 raft_consensus.cc:2258] T a0ad134e0bab43d6b9a63b04e05e9b97 P a47af5c80dac43b19d033adba1a62d4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:46:59.940635 28720 ts_tablet_manager.cc:1905] T a0ad134e0bab43d6b9a63b04e05e9b97 P a47af5c80dac43b19d033adba1a62d4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:46:59.944931 28718 ts_tablet_manager.cc:1918] T a0ad134e0bab43d6b9a63b04e05e9b97 P ca00b8cfda2742ff9e79c74f2c62a843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:46:59.945088 28718 log.cc:1196] T a0ad134e0bab43d6b9a63b04e05e9b97 P ca00b8cfda2742ff9e79c74f2c62a843: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/wal/wals/a0ad134e0bab43d6b9a63b04e05e9b97
I20260809 04:46:59.945647 28718 ts_tablet_manager.cc:1939] T a0ad134e0bab43d6b9a63b04e05e9b97 P ca00b8cfda2742ff9e79c74f2c62a843: Deleting consensus metadata
I20260809 04:46:59.946542 28719 tablet_replica.cc:330] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828: stopping tablet replica
I20260809 04:46:59.947477 28145 catalog_manager.cc:4841] TS ca00b8cfda2742ff9e79c74f2c62a843 (127.12.234.3:45063): tablet a0ad134e0bab43d6b9a63b04e05e9b97 (table test-table [id=c4ce41aa9e894098af007fbc29393da8]) successfully deleted
I20260809 04:46:59.947983 28719 raft_consensus.cc:2229] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:46:59.948683 28719 raft_consensus.cc:2258] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 09 04:46:59 dist-test-slave-1jjb krb5kdc[28107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.1@KRBTEST.COM
I20260809 04:46:59.951232 28719 ts_tablet_manager.cc:1905] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:46:59.951471 28720 ts_tablet_manager.cc:1918] T a0ad134e0bab43d6b9a63b04e05e9b97 P a47af5c80dac43b19d033adba1a62d4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:46:59.951613 28720 log.cc:1196] T a0ad134e0bab43d6b9a63b04e05e9b97 P a47af5c80dac43b19d033adba1a62d4b: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/wal/wals/a0ad134e0bab43d6b9a63b04e05e9b97
I20260809 04:46:59.952152 28720 ts_tablet_manager.cc:1939] T a0ad134e0bab43d6b9a63b04e05e9b97 P a47af5c80dac43b19d033adba1a62d4b: Deleting consensus metadata
I20260809 04:46:59.953727 28143 catalog_manager.cc:4841] TS a47af5c80dac43b19d033adba1a62d4b (127.12.234.1:44033): tablet a0ad134e0bab43d6b9a63b04e05e9b97 (table test-table [id=c4ce41aa9e894098af007fbc29393da8]) successfully deleted
I20260809 04:46:59.958350 28719 ts_tablet_manager.cc:1918] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:46:59.958528 28719 log.cc:1196] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/wal/wals/a0ad134e0bab43d6b9a63b04e05e9b97
I20260809 04:46:59.959146 28719 ts_tablet_manager.cc:1939] T a0ad134e0bab43d6b9a63b04e05e9b97 P 2a69dc7a46a140f49df77a742cc1c828: Deleting consensus metadata
I20260809 04:46:59.960156 28690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:46:59.924014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59816 (local address 127.12.234.1:44033)
0809 04:46:59.924258 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 04:46:59.924266 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:46:59.931108 (+ 6842us) server_negotiation.cc:407] Connection header received
0809 04:46:59.932092 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:46:59.932099 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:46:59.932184 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:46:59.932267 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:46:59.938237 (+ 5970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.939370 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.944181 (+ 4811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:46:59.944800 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:46:59.944929 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:46:59.952455 (+ 7526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:46:59.952468 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:46:59.952484 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:46:59.952524 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:46:59.956635 (+ 4111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:59.957482 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:59.957491 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:59.957495 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:59.957645 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:46:59.958083 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:46:59.958091 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:46:59.958097 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:46:59.958548 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:46:59.958877 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0809 04:46:59.959507 (+ 630us) server_negotiation.cc:299] Negotiation successful
0809 04:46:59.959719 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 04:46:59.960866 28361 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:46:59.960808 28144 catalog_manager.cc:4841] TS 2a69dc7a46a140f49df77a742cc1c828 (127.12.234.2:45601): tablet a0ad134e0bab43d6b9a63b04e05e9b97 (table test-table [id=c4ce41aa9e894098af007fbc29393da8]) successfully deleted
I20260809 04:47:00.055683 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 28227
I20260809 04:47:00.077819 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 28374
Aug 09 04:47:00 dist-test-slave-1jjb krb5kdc[28107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.2@KRBTEST.COM
I20260809 04:47:00.098234 28688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:00.078598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45620 (local address 127.12.234.2:45601)
0809 04:47:00.078779 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:00.078787 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:00.078807 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:47:00.079015 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:00.079019 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:00.079083 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:00.079186 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:00.080651 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:00.081824 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:00.083127 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:00.083587 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:00.083723 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:00.092582 (+ 8859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:00.092592 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:00.092605 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:00.092644 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:00.095543 (+ 2899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:00.096320 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:00.096334 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:00.096336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:00.096473 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:00.096836 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:00.096839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:00.096841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:00.097167 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:00.097485 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:00.097701 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 04:47:00.097912 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 04:47:00.098886 28515 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:47:00.208276 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 28374
I20260809 04:47:00.229786 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 28526
Aug 09 04:47:00 dist-test-slave-1jjb krb5kdc[28107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.3@KRBTEST.COM
I20260809 04:47:00.243911 28687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:00.230578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45558 (local address 127.12.234.3:45063)
0809 04:47:00.230764 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:00.230768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:00.230787 (+ 19us) server_negotiation.cc:407] Connection header received
0809 04:47:00.230906 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:00.230911 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:00.230984 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:00.231067 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:00.232417 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:00.233126 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:00.234303 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:00.234656 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:00.234738 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:00.238631 (+ 3893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:00.238644 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:00.238660 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:00.238688 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:00.241428 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:00.242045 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:00.242052 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:00.242054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:00.242207 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:00.242623 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:00.242626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:00.242628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:00.242998 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:00.243197 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:00.243519 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 04:47:00.243681 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 04:47:00.244634 28667 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:47:00.327505 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 28526
I20260809 04:47:00.347025 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 28121
I20260809 04:47:00.348037 28191 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:47:00.452425 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 28121
I20260809 04:47:00.478193 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:44829
--webserver_interface=127.12.234.62
--webserver_port=36611
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:44829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:47:00.787086 28728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:00.787374 28728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:00.787524 28728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:00.794255 28728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:47:00.794399 28728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:00.794499 28728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:00.794591 28728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:47:00.794672 28728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:47:00.803892 28728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:44829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:44829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=36611
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:47:00.805017 28728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:00.806115 28728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:00.833446 28737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:00.833469 28736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:00.833514 28739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:00.834273 28728 server_base.cc:1034] running on GCE node
I20260809 04:47:00.835101 28728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:00.836670 28728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:00.837901 28728 hybrid_clock.cc:648] HybridClock initialized: now 1786250820837847 us; error 59 us; skew 500 ppm
I20260809 04:47:00.838251 28728 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:47:00 dist-test-slave-1jjb krb5kdc[28107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250820, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:00.841570 28728 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:47:00.843080 28728 webserver.cc:460] Webserver started at http://127.12.234.62:36611/ using document root <none> and password file <none>
I20260809 04:47:00.843824 28728 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:00.843998 28728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:00.849529 28728 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:47:00.852402 28748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:00.853662 28728 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:47:00.853852 28728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "03dfd2bb88764bb9b513bf65e731fbca"
format_stamp: "Formatted at 2026-08-09 04:46:57 on dist-test-slave-1jjb"
server_key: "8ac237defb469ddc13cc830fb16d9b56"
server_key_iv: "57fc15cfc39ce527d14eb7a3450eca20"
server_key_version: "encryptionkey@0"
I20260809 04:47:00.854833 28728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:00.874382 28728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:00.878064 28728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:00.878343 28728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:00.897475 28728 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:44829
I20260809 04:47:00.897513 28811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:44829 every 8 connection(s)
I20260809 04:47:00.898818 28728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:47:00.902900 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 28728
I20260809 04:47:00.903682 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:44033
--local_ip_for_outbound_sockets=127.12.234.1
--tserver_master_addrs=127.12.234.62:44829
--webserver_port=40639
--webserver_interface=127.12.234.1
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:47:00.906016 28812 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:47:00.909137 28812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca: Bootstrap starting.
I20260809 04:47:00.928728 28812 log.cc:824] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:00.939723 28812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:47:00.940364 28812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca: Bootstrap complete.
I20260809 04:47:00.948773 28812 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03dfd2bb88764bb9b513bf65e731fbca" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44829 } }
I20260809 04:47:00.949573 28812 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03dfd2bb88764bb9b513bf65e731fbca, State: Initialized, Role: FOLLOWER
I20260809 04:47:00.950208 28812 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03dfd2bb88764bb9b513bf65e731fbca" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44829 } }
I20260809 04:47:00.950469 28812 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:47:00.950608 28812 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:47:00.950793 28812 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [term 1 FOLLOWER]: Advancing to term 2
I20260809 04:47:00.954098 28812 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03dfd2bb88764bb9b513bf65e731fbca" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44829 } }
I20260809 04:47:00.954555 28812 leader_election.cc:302] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03dfd2bb88764bb9b513bf65e731fbca; no voters:
I20260809 04:47:00.955152 28812 leader_election.cc:288] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 04:47:00.955277 28817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [term 2 FOLLOWER]: Leader election won for term 2
I20260809 04:47:00.956615 28817 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [term 2 LEADER]: Becoming Leader. State: Replica: 03dfd2bb88764bb9b513bf65e731fbca, State: Running, Role: LEADER
I20260809 04:47:00.957109 28817 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03dfd2bb88764bb9b513bf65e731fbca" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44829 } }
I20260809 04:47:00.957342 28812 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:47:00.959800 28819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03dfd2bb88764bb9b513bf65e731fbca. Latest consensus state: current_term: 2 leader_uuid: "03dfd2bb88764bb9b513bf65e731fbca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03dfd2bb88764bb9b513bf65e731fbca" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44829 } } }
I20260809 04:47:00.959808 28818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "03dfd2bb88764bb9b513bf65e731fbca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03dfd2bb88764bb9b513bf65e731fbca" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44829 } } }
I20260809 04:47:00.960330 28819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [sys.catalog]: This master's current role is: LEADER
I20260809 04:47:00.960410 28818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [sys.catalog]: This master's current role is: LEADER
I20260809 04:47:00.970497 28822 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:47:00.978145 28822 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:47:00.979602 28822 catalog_manager.cc:1259] Loaded cluster ID: 76c454d5ebc84f08a710b36bf9389929
I20260809 04:47:00.979688 28822 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:47:00.982766 28822 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 04:47:01.000111 28822 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 04:47:01.015671 28822 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:47:01.017313 28822 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca: Loaded TSK: 0
I20260809 04:47:01.018720 28822 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 04:47:01.245537 28814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:01.245841 28814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:01.246021 28814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:01.253016 28814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:01.253165 28814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:01.253340 28814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:47:01.262694 28814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:44033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=40639
--tserver_master_addrs=127.12.234.62:44829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:47:01.263906 28814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:01.265230 28814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:01.279063 28846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:01.279239 28847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:01.279932 28849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:01.280359 28814 server_base.cc:1034] running on GCE node
I20260809 04:47:01.280951 28814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:01.281873 28814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:01.283108 28814 hybrid_clock.cc:648] HybridClock initialized: now 1786250821283050 us; error 49 us; skew 500 ppm
I20260809 04:47:01.283411 28814 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:47:01 dist-test-slave-1jjb krb5kdc[28107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250821, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:01.286466 28814 init.cc:377] Logged in from keytab as kudu/127.12.234.1@KRBTEST.COM (short username kudu)
I20260809 04:47:01.287914 28814 webserver.cc:460] Webserver started at http://127.12.234.1:40639/ using document root <none> and password file <none>
I20260809 04:47:01.288626 28814 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:01.288785 28814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:01.294438 28814 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 04:47:01.297106 28856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:01.298499 28814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:47:01.298642 28814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "a47af5c80dac43b19d033adba1a62d4b"
format_stamp: "Formatted at 2026-08-09 04:46:58 on dist-test-slave-1jjb"
server_key: "8f92676e2861bac6ef2bd56ea15ca845"
server_key_iv: "bc129a89fdc895567f2f6c14cf942498"
server_key_version: "encryptionkey@0"
I20260809 04:47:01.299723 28814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:01.318661 28814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:01.322225 28814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:01.322433 28814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:01.323205 28814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:47:01.324469 28814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:47:01.324576 28814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:01.324661 28814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:47:01.324715 28814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:01.355785 28814 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:44033
I20260809 04:47:01.355885 28985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:44033 every 8 connection(s)
I20260809 04:47:01.357200 28814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:47:01.359484 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 28814
I20260809 04:47:01.360298 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:45601
--local_ip_for_outbound_sockets=127.12.234.2
--tserver_master_addrs=127.12.234.62:44829
--webserver_port=39859
--webserver_interface=127.12.234.2
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:47:01 dist-test-slave-1jjb krb5kdc[28107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250821, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:47:01.379792 28989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:01.360117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:52071 (local address 127.12.234.62:44829)
0809 04:47:01.360737 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:01.360752 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:01.361452 (+ 700us) server_negotiation.cc:407] Connection header received
0809 04:47:01.363425 (+ 1973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:01.363437 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:01.363705 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:01.364047 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:01.365427 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:01.366510 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:01.367905 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:01.368582 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:01.368766 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:01.373049 (+ 4283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:01.373056 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:01.373069 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:01.373102 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:01.375855 (+ 2753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:01.376751 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:01.376757 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:01.376762 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:01.376922 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:01.377389 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:01.377392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:01.377394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:01.377763 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:01.377900 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:01.378713 (+ 813us) server_negotiation.cc:299] Negotiation successful
0809 04:47:01.378982 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":399,"threads_started":1}
I20260809 04:47:01.380327 28990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:01.360496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.1:52071)
0809 04:47:01.361263 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:01.361298 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:01.362903 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:01.364202 (+ 1299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:01.364215 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:01.364776 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:01.365215 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:01.365229 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:01.366674 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:01.366678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:01.367737 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:01.367748 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:01.368813 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:01.368820 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:01.369078 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:01.369755 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:01.369794 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:01.372750 (+ 2956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:01.375989 (+ 3239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:01.375999 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:01.376014 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:01.376628 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:01.377033 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:01.377040 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:01.377046 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:01.377283 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:01.377899 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:01.377908 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:01.378300 (+ 392us) client_negotiation.cc:769] Sending connection context
0809 04:47:01.378557 (+ 257us) client_negotiation.cc:240] Negotiation successful
0809 04:47:01.379006 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":326,"threads_started":1}
I20260809 04:47:01.383019 28987 heartbeater.cc:347] Connected to a master server at 127.12.234.62:44829
I20260809 04:47:01.383365 28987 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:01.384325 28987 heartbeater.cc:511] Master 127.12.234.62:44829 requested a full tablet report, sending...
I20260809 04:47:01.386550 28770 ts_manager.cc:194] Registered new tserver with Master: a47af5c80dac43b19d033adba1a62d4b (127.12.234.1:44033)
I20260809 04:47:01.387918 28770 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.1@KRBTEST.COM'} at 127.12.234.1:52071
W20260809 04:47:01.680526 28991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:01.680757 28991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:01.680950 28991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:01.687681 28991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:01.687791 28991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:01.687961 28991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:47:01.697214 28991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:45601
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=39859
--tserver_master_addrs=127.12.234.62:44829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:47:01.698271 28991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:01.699527 28991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:01.711823 28999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:01.711871 28998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:01.711871 29001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:01.712988 28991 server_base.cc:1034] running on GCE node
I20260809 04:47:01.713505 28991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:01.714285 28991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:01.715520 28991 hybrid_clock.cc:648] HybridClock initialized: now 1786250821715466 us; error 43 us; skew 500 ppm
I20260809 04:47:01.715797 28991 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:47:01 dist-test-slave-1jjb krb5kdc[28107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250821, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:01.718636 28991 init.cc:377] Logged in from keytab as kudu/127.12.234.2@KRBTEST.COM (short username kudu)
I20260809 04:47:01.719882 28991 webserver.cc:460] Webserver started at http://127.12.234.2:39859/ using document root <none> and password file <none>
I20260809 04:47:01.720608 28991 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:01.720774 28991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:01.725584 28991 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260809 04:47:01.728032 29008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:01.729277 28991 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:47:01.729447 28991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "2a69dc7a46a140f49df77a742cc1c828"
format_stamp: "Formatted at 2026-08-09 04:46:59 on dist-test-slave-1jjb"
server_key: "8fe6ece157b8fe538c15637314298f89"
server_key_iv: "f0e39962459b4d96cca01c76fb180c80"
server_key_version: "encryptionkey@0"
I20260809 04:47:01.730367 28991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:01.742573 28991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:01.745738 28991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:01.745980 28991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:01.746729 28991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:47:01.747810 28991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:47:01.747917 28991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:01.748023 28991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:47:01.748114 28991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:01.779332 28991 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:45601
I20260809 04:47:01.779382 29133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:45601 every 8 connection(s)
I20260809 04:47:01.780625 28991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:47:01.788595 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 28991
I20260809 04:47:01.789299 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:45063
--local_ip_for_outbound_sockets=127.12.234.3
--tserver_master_addrs=127.12.234.62:44829
--webserver_port=44661
--webserver_interface=127.12.234.3
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:47:01 dist-test-slave-1jjb krb5kdc[28107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250821, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:47:01.800457 28989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:01.783054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:58519 (local address 127.12.234.62:44829)
0809 04:47:01.783249 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:01.783253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:01.784362 (+ 1109us) server_negotiation.cc:407] Connection header received
0809 04:47:01.786125 (+ 1763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:01.786129 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:01.786192 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:01.786273 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:01.787499 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:01.788177 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:01.790577 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:01.790991 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:01.791093 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:01.795049 (+ 3956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:01.795053 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:01.795056 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:01.795084 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:01.797656 (+ 2572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:01.798296 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:01.798302 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:01.798305 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:01.798432 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:01.798827 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:01.798832 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:01.798835 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:01.799085 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:01.799252 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:01.799836 (+ 584us) server_negotiation.cc:299] Negotiation successful
0809 04:47:01.800063 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 04:47:01.801275 29137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:01.783507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.2:58519)
0809 04:47:01.784218 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:01.784253 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:01.785899 (+ 1646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:01.786449 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:01.786455 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:01.786858 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:01.787363 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:01.787380 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:01.788316 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:01.788333 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:01.790348 (+ 2015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:01.790373 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:01.791215 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:01.791225 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:01.791530 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:01.792181 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:01.792209 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:01.794831 (+ 2622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:01.797798 (+ 2967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:01.797805 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:01.797817 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:01.798181 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:01.798537 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:01.798541 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:01.798545 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:01.798711 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:01.799225 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:01.799232 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:01.799508 (+ 276us) client_negotiation.cc:769] Sending connection context
0809 04:47:01.799743 (+ 235us) client_negotiation.cc:240] Negotiation successful
0809 04:47:01.800244 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":261,"threads_started":1}
I20260809 04:47:01.802708 29135 heartbeater.cc:347] Connected to a master server at 127.12.234.62:44829
I20260809 04:47:01.803002 29135 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:01.803669 29135 heartbeater.cc:511] Master 127.12.234.62:44829 requested a full tablet report, sending...
I20260809 04:47:01.805188 28770 ts_manager.cc:194] Registered new tserver with Master: 2a69dc7a46a140f49df77a742cc1c828 (127.12.234.2:45601)
I20260809 04:47:01.805953 28770 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.2@KRBTEST.COM'} at 127.12.234.2:58519
W20260809 04:47:02.108603 29138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:02.108850 29138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:02.109001 29138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:02.115675 29138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:02.115801 29138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:02.115962 29138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:47:02.124600 29138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:45063
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=44661
--tserver_master_addrs=127.12.234.62:44829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:47:02.125622 29138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:02.126802 29138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:02.138974 29143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:02.139000 29144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:02.139557 29147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:02.139530 29138 server_base.cc:1034] running on GCE node
I20260809 04:47:02.140273 29138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:02.141119 29138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:02.142333 29138 hybrid_clock.cc:648] HybridClock initialized: now 1786250822142293 us; error 56 us; skew 500 ppm
I20260809 04:47:02.142627 29138 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:47:02 dist-test-slave-1jjb krb5kdc[28107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250822, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:02.145294 29138 init.cc:377] Logged in from keytab as kudu/127.12.234.3@KRBTEST.COM (short username kudu)
I20260809 04:47:02.146555 29138 webserver.cc:460] Webserver started at http://127.12.234.3:44661/ using document root <none> and password file <none>
I20260809 04:47:02.147274 29138 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:02.147418 29138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:02.152066 29138 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:47:02.154420 29154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:02.155494 29138 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:47:02.155668 29138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "ca00b8cfda2742ff9e79c74f2c62a843"
format_stamp: "Formatted at 2026-08-09 04:46:59 on dist-test-slave-1jjb"
server_key: "38339b9d93170199d348fe64d6b41f5c"
server_key_iv: "6424c17a5cb5be714b43c3870b615102"
server_key_version: "encryptionkey@0"
I20260809 04:47:02.156599 29138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:02.178288 29138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:02.181705 29138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:02.181972 29138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:02.182807 29138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:47:02.184056 29138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:47:02.184209 29138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:02.184340 29138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:47:02.184437 29138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:02.218503 29138 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:45063
I20260809 04:47:02.218583 29280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:45063 every 8 connection(s)
I20260809 04:47:02.219900 29138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:47:02.227772 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 29138
Aug 09 04:47:02 dist-test-slave-1jjb krb5kdc[28107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250822, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:47:02.240597 28989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:02.222564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:54109 (local address 127.12.234.62:44829)
0809 04:47:02.222749 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:02.222755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:02.223746 (+ 991us) server_negotiation.cc:407] Connection header received
0809 04:47:02.225426 (+ 1680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:02.225431 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:02.225497 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:02.225578 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:02.226908 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.227621 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.229845 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.230556 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.230648 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:02.234389 (+ 3741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:02.234396 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:02.234400 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:02.234437 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:02.237266 (+ 2829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:02.238092 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:02.238097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:02.238100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:02.238270 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:02.238675 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:02.238682 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:02.238687 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:02.239031 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:02.239181 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:02.239942 (+ 761us) server_negotiation.cc:299] Negotiation successful
0809 04:47:02.240201 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 04:47:02.241508 29283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:02.222926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.3:54109)
0809 04:47:02.223575 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:02.223617 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:02.225195 (+ 1578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:02.225771 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:02.225778 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:02.226215 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:02.226732 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:02.226748 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.227789 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.227797 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:02.229685 (+ 1888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:02.229702 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.230812 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.230822 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:02.231139 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:02.231844 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:02.231881 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:02.234192 (+ 2311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:02.237407 (+ 3215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:02.237416 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:02.237430 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:02.237978 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:02.238383 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:02.238386 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:02.238388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:02.238548 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:02.239185 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:02.239191 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:02.239467 (+ 276us) client_negotiation.cc:769] Sending connection context
0809 04:47:02.239820 (+ 353us) client_negotiation.cc:240] Negotiation successful
0809 04:47:02.240268 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":244,"threads_started":1}
I20260809 04:47:02.242985 29281 heartbeater.cc:347] Connected to a master server at 127.12.234.62:44829
I20260809 04:47:02.243280 29281 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:02.244027 29284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:02.228713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55394 (local address 127.12.234.62:44829)
0809 04:47:02.229386 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:02.229394 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:02.229946 (+ 552us) server_negotiation.cc:407] Connection header received
0809 04:47:02.230061 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:02.230066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:02.230128 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:02.230243 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:02.232312 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.233006 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.234519 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.235137 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.235272 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:02.238101 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:02.238106 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:02.238110 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:02.238139 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:02.241176 (+ 3037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:02.242013 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:02.242020 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:02.242024 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:02.242170 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:02.242790 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:02.242793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:02.242795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:02.243024 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:02.243238 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:02.243649 (+ 411us) server_negotiation.cc:299] Negotiation successful
0809 04:47:02.243779 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":592,"threads_started":1}
I20260809 04:47:02.244251 29281 heartbeater.cc:511] Master 127.12.234.62:44829 requested a full tablet report, sending...
I20260809 04:47:02.245831 28769 ts_manager.cc:194] Registered new tserver with Master: ca00b8cfda2742ff9e79c74f2c62a843 (127.12.234.3:45063)
I20260809 04:47:02.246801 28769 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.3@KRBTEST.COM'} at 127.12.234.3:54109
I20260809 04:47:02.258358 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:47:02.264750 29284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:02.259316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55404 (local address 127.12.234.62:44829)
0809 04:47:02.259540 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:02.259545 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:02.259811 (+ 266us) server_negotiation.cc:407] Connection header received
0809 04:47:02.260018 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:02.260022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:02.260191 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:02.260352 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:47:02.260757 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.261408 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.262761 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.263164 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.263301 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:02.263754 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:47:02.264229 (+ 475us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:47:02.264364 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:02.264464 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 04:47:02.264536 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:47:02.265543 28769 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:55404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:47:02.271986 28769 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:47:02.301968 29291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:02.281979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:45601 (local address 127.0.0.1:57094)
0809 04:47:02.293189 (+ 11210us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:02.293222 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:02.293405 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:02.294352 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:02.294359 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:02.294389 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:02.294958 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:02.294972 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.296894 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.296901 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:02.298921 (+ 2020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:02.298936 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.301041 (+ 2105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.301045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:02.301255 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:02.301325 (+ 70us) client_negotiation.cc:769] Sending connection context
0809 04:47:02.301462 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 04:47:02.301644 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10999,"thread_start_us":10913,"threads_started":1}
I20260809 04:47:02.302297 29295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:02.282726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37526 (local address 127.12.234.3:45063)
0809 04:47:02.292848 (+ 10122us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:02.292858 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:02.294684 (+ 1826us) server_negotiation.cc:407] Connection header received
0809 04:47:02.294776 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:02.294784 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:02.294990 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:02.295169 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:02.295991 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.297349 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.299570 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.301017 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.301156 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:02.301394 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:02.301746 (+ 352us) server_negotiation.cc:299] Negotiation successful
0809 04:47:02.301888 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9928,"thread_start_us":9803,"threads_started":1}
I20260809 04:47:02.302488 29294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:02.282093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:45063 (local address 127.0.0.1:37526)
0809 04:47:02.294413 (+ 12320us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:02.294430 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:02.294555 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:02.295453 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:02.295457 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:02.295481 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:02.295812 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:02.295822 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.297501 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.297506 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:02.299430 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:02.299440 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.301231 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.301238 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:02.301442 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:02.301478 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 04:47:02.301564 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 04:47:02.301654 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12183,"mutex_wait_us":2,"thread_start_us":12550,"threads_started":1}
I20260809 04:47:02.302479 29293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:02.282254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57094 (local address 127.12.234.2:45601)
0809 04:47:02.293656 (+ 11402us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:02.293666 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:02.293701 (+ 35us) server_negotiation.cc:407] Connection header received
0809 04:47:02.293790 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:02.293798 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:02.294035 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:02.295170 (+ 1135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:02.295454 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.296773 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.299085 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.300602 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.300726 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:02.300926 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:02.301549 (+ 623us) server_negotiation.cc:299] Negotiation successful
0809 04:47:02.301702 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11234,"thread_start_us":10087,"threads_started":1}
I20260809 04:47:02.303052 29296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:02.282994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35032 (local address 127.12.234.1:44033)
0809 04:47:02.293908 (+ 10914us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:02.293916 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:02.295226 (+ 1310us) server_negotiation.cc:407] Connection header received
0809 04:47:02.295329 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:02.295336 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:02.295550 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:02.295744 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:02.296513 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.297762 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.299692 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.301137 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.301257 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:02.301418 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:02.302597 (+ 1179us) server_negotiation.cc:299] Negotiation successful
0809 04:47:02.302751 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10770,"thread_start_us":10667,"threads_started":1}
I20260809 04:47:02.303625 29292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:02.282458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:44033 (local address 127.0.0.1:35032)
0809 04:47:02.294958 (+ 12500us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:02.294976 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:02.295104 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:02.295977 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:02.295981 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:02.296003 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:02.296338 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:02.296348 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.297883 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.297890 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:02.299526 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:02.299542 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.301602 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.301610 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:02.302094 (+ 484us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:02.302123 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 04:47:02.302209 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 04:47:02.302340 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12328,"mutex_wait_us":30,"thread_start_us":11691,"threads_started":1}
I20260809 04:47:02.305886 29206 tablet_service.cc:1467] Processing CreateTablet for tablet 3acdad44042f4e5888dbe9fc4c066743 (DEFAULT_TABLE table=test-table [id=cdc1f9815083413fb219cd0e93eda55e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:47:02.306043 29062 tablet_service.cc:1467] Processing CreateTablet for tablet 3acdad44042f4e5888dbe9fc4c066743 (DEFAULT_TABLE table=test-table [id=cdc1f9815083413fb219cd0e93eda55e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:47:02.307240 29206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3acdad44042f4e5888dbe9fc4c066743. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:47:02.307278 29062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3acdad44042f4e5888dbe9fc4c066743. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:47:02.306983 28915 tablet_service.cc:1467] Processing CreateTablet for tablet 3acdad44042f4e5888dbe9fc4c066743 (DEFAULT_TABLE table=test-table [id=cdc1f9815083413fb219cd0e93eda55e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:47:02.308203 28915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3acdad44042f4e5888dbe9fc4c066743. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:47:02.318184 29297 tablet_bootstrap.cc:492] T 3acdad44042f4e5888dbe9fc4c066743 P ca00b8cfda2742ff9e79c74f2c62a843: Bootstrap starting.
I20260809 04:47:02.321441 29297 tablet_bootstrap.cc:654] T 3acdad44042f4e5888dbe9fc4c066743 P ca00b8cfda2742ff9e79c74f2c62a843: Neither blocks nor log segments found. Creating new log.
I20260809 04:47:02.321779 29298 tablet_bootstrap.cc:492] T 3acdad44042f4e5888dbe9fc4c066743 P 2a69dc7a46a140f49df77a742cc1c828: Bootstrap starting.
I20260809 04:47:02.322634 29299 tablet_bootstrap.cc:492] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b: Bootstrap starting.
I20260809 04:47:02.323954 29297 log.cc:824] T 3acdad44042f4e5888dbe9fc4c066743 P ca00b8cfda2742ff9e79c74f2c62a843: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:02.325356 29298 tablet_bootstrap.cc:654] T 3acdad44042f4e5888dbe9fc4c066743 P 2a69dc7a46a140f49df77a742cc1c828: Neither blocks nor log segments found. Creating new log.
I20260809 04:47:02.325556 29299 tablet_bootstrap.cc:654] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b: Neither blocks nor log segments found. Creating new log.
I20260809 04:47:02.326534 29299 log.cc:824] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:02.326576 29298 log.cc:824] T 3acdad44042f4e5888dbe9fc4c066743 P 2a69dc7a46a140f49df77a742cc1c828: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:02.327032 29297 tablet_bootstrap.cc:492] T 3acdad44042f4e5888dbe9fc4c066743 P ca00b8cfda2742ff9e79c74f2c62a843: No bootstrap required, opened a new log
I20260809 04:47:02.327374 29297 ts_tablet_manager.cc:1397] T 3acdad44042f4e5888dbe9fc4c066743 P ca00b8cfda2742ff9e79c74f2c62a843: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 04:47:02.329015 29299 tablet_bootstrap.cc:492] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b: No bootstrap required, opened a new log
I20260809 04:47:02.329278 29299 ts_tablet_manager.cc:1397] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 04:47:02.333894 29298 tablet_bootstrap.cc:492] T 3acdad44042f4e5888dbe9fc4c066743 P 2a69dc7a46a140f49df77a742cc1c828: No bootstrap required, opened a new log
I20260809 04:47:02.334125 29298 ts_tablet_manager.cc:1397] T 3acdad44042f4e5888dbe9fc4c066743 P 2a69dc7a46a140f49df77a742cc1c828: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.000s
I20260809 04:47:02.337144 29299 raft_consensus.cc:348] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69dc7a46a140f49df77a742cc1c828" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45601 } } peers { permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 } } peers { permanent_uuid: "a47af5c80dac43b19d033adba1a62d4b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 44033 } }
I20260809 04:47:02.337491 29299 raft_consensus.cc:374] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:47:02.337616 29299 raft_consensus.cc:729] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a47af5c80dac43b19d033adba1a62d4b, State: Initialized, Role: FOLLOWER
I20260809 04:47:02.338186 29299 consensus_queue.cc:260] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69dc7a46a140f49df77a742cc1c828" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45601 } } peers { permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 } } peers { permanent_uuid: "a47af5c80dac43b19d033adba1a62d4b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 44033 } }
I20260809 04:47:02.339238 28987 heartbeater.cc:503] Master 127.12.234.62:44829 was elected leader, sending a full tablet report...
I20260809 04:47:02.339193 29297 raft_consensus.cc:348] T 3acdad44042f4e5888dbe9fc4c066743 P ca00b8cfda2742ff9e79c74f2c62a843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69dc7a46a140f49df77a742cc1c828" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45601 } } peers { permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 } } peers { permanent_uuid: "a47af5c80dac43b19d033adba1a62d4b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 44033 } }
I20260809 04:47:02.339596 29297 raft_consensus.cc:374] T 3acdad44042f4e5888dbe9fc4c066743 P ca00b8cfda2742ff9e79c74f2c62a843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:47:02.339697 29297 raft_consensus.cc:729] T 3acdad44042f4e5888dbe9fc4c066743 P ca00b8cfda2742ff9e79c74f2c62a843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca00b8cfda2742ff9e79c74f2c62a843, State: Initialized, Role: FOLLOWER
I20260809 04:47:02.339828 29299 ts_tablet_manager.cc:1428] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260809 04:47:02.340378 29297 consensus_queue.cc:260] T 3acdad44042f4e5888dbe9fc4c066743 P ca00b8cfda2742ff9e79c74f2c62a843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69dc7a46a140f49df77a742cc1c828" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45601 } } peers { permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 } } peers { permanent_uuid: "a47af5c80dac43b19d033adba1a62d4b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 44033 } }
I20260809 04:47:02.341444 29281 heartbeater.cc:503] Master 127.12.234.62:44829 was elected leader, sending a full tablet report...
I20260809 04:47:02.343091 29297 ts_tablet_manager.cc:1428] T 3acdad44042f4e5888dbe9fc4c066743 P ca00b8cfda2742ff9e79c74f2c62a843: Time spent starting tablet: real 0.016s user 0.010s sys 0.004s
I20260809 04:47:02.346267 29298 raft_consensus.cc:348] T 3acdad44042f4e5888dbe9fc4c066743 P 2a69dc7a46a140f49df77a742cc1c828 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69dc7a46a140f49df77a742cc1c828" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45601 } } peers { permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 } } peers { permanent_uuid: "a47af5c80dac43b19d033adba1a62d4b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 44033 } }
I20260809 04:47:02.346558 29298 raft_consensus.cc:374] T 3acdad44042f4e5888dbe9fc4c066743 P 2a69dc7a46a140f49df77a742cc1c828 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:47:02.346663 29298 raft_consensus.cc:729] T 3acdad44042f4e5888dbe9fc4c066743 P 2a69dc7a46a140f49df77a742cc1c828 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a69dc7a46a140f49df77a742cc1c828, State: Initialized, Role: FOLLOWER
I20260809 04:47:02.347121 29298 consensus_queue.cc:260] T 3acdad44042f4e5888dbe9fc4c066743 P 2a69dc7a46a140f49df77a742cc1c828 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69dc7a46a140f49df77a742cc1c828" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45601 } } peers { permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 } } peers { permanent_uuid: "a47af5c80dac43b19d033adba1a62d4b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 44033 } }
I20260809 04:47:02.348122 29135 heartbeater.cc:503] Master 127.12.234.62:44829 was elected leader, sending a full tablet report...
I20260809 04:47:02.348584 29298 ts_tablet_manager.cc:1428] T 3acdad44042f4e5888dbe9fc4c066743 P 2a69dc7a46a140f49df77a742cc1c828: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
W20260809 04:47:02.359292 28988 tablet.cc:2369] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:47:02.392442 29303 raft_consensus.cc:482] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:47:02.392670 29303 raft_consensus.cc:504] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69dc7a46a140f49df77a742cc1c828" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45601 } } peers { permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 } } peers { permanent_uuid: "a47af5c80dac43b19d033adba1a62d4b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 44033 } }
I20260809 04:47:02.394097 29303 leader_election.cc:288] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a69dc7a46a140f49df77a742cc1c828 (127.12.234.2:45601), ca00b8cfda2742ff9e79c74f2c62a843 (127.12.234.3:45063)
I20260809 04:47:02.400885 29293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:02.394538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:46451 (local address 127.12.234.2:45601)
0809 04:47:02.394730 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:02.394736 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:02.395738 (+ 1002us) server_negotiation.cc:407] Connection header received
0809 04:47:02.395916 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:02.395921 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:02.395990 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:02.396104 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:02.396719 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.397352 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.399154 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.399896 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.400011 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:02.400135 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:02.400491 (+ 356us) server_negotiation.cc:299] Negotiation successful
0809 04:47:02.400622 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:47:02.400921 29295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:02.394500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:48173 (local address 127.12.234.3:45063)
0809 04:47:02.394706 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:02.394712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:02.395529 (+ 817us) server_negotiation.cc:407] Connection header received
0809 04:47:02.395720 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:02.395726 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:02.395796 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:02.395899 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:02.396544 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.397271 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.399196 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.399934 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.400146 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:02.400215 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:02.400548 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 04:47:02.400677 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 04:47:02.401012 29307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:02.394593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:45601 (local address 127.12.234.1:46451)
0809 04:47:02.395607 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:02.395640 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:02.395772 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:02.396170 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:02.396174 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:02.396192 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:02.396544 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:02.396566 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.397472 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.397478 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:02.398977 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:02.398987 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.400122 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.400131 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:02.400333 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:02.400357 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 04:47:02.400432 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 04:47:02.400566 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":890,"mutex_wait_us":29,"spinlock_wait_cycles":7424,"thread_start_us":677,"threads_started":1}
I20260809 04:47:02.401036 29306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:02.394584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:45063 (local address 127.12.234.1:48173)
0809 04:47:02.395389 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:02.395412 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:02.395563 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:02.395925 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:02.395930 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:02.395953 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:02.396399 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:02.396413 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.397429 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.397436 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:02.398982 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:02.399001 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.400214 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.400218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:02.400368 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:02.400390 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 04:47:02.400506 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 04:47:02.400621 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"mutex_wait_us":111,"thread_start_us":434,"threads_started":1}
I20260809 04:47:02.401906 29226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3acdad44042f4e5888dbe9fc4c066743" candidate_uuid: "a47af5c80dac43b19d033adba1a62d4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" is_pre_election: true
I20260809 04:47:02.401939 29083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3acdad44042f4e5888dbe9fc4c066743" candidate_uuid: "a47af5c80dac43b19d033adba1a62d4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a69dc7a46a140f49df77a742cc1c828" is_pre_election: true
I20260809 04:47:02.402340 29083 raft_consensus.cc:2454] T 3acdad44042f4e5888dbe9fc4c066743 P 2a69dc7a46a140f49df77a742cc1c828 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a47af5c80dac43b19d033adba1a62d4b in term 0.
I20260809 04:47:02.402338 29226 raft_consensus.cc:2454] T 3acdad44042f4e5888dbe9fc4c066743 P ca00b8cfda2742ff9e79c74f2c62a843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a47af5c80dac43b19d033adba1a62d4b in term 0.
I20260809 04:47:02.402998 28859 leader_election.cc:302] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a69dc7a46a140f49df77a742cc1c828, a47af5c80dac43b19d033adba1a62d4b; no voters:
I20260809 04:47:02.403376 29303 raft_consensus.cc:2781] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:47:02.403523 29303 raft_consensus.cc:482] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:47:02.403676 29303 raft_consensus.cc:3037] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:47:02.406136 29303 raft_consensus.cc:504] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69dc7a46a140f49df77a742cc1c828" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45601 } } peers { permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 } } peers { permanent_uuid: "a47af5c80dac43b19d033adba1a62d4b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 44033 } }
I20260809 04:47:02.406589 29303 leader_election.cc:288] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [CANDIDATE]: Term 1 election: Requested vote from peers 2a69dc7a46a140f49df77a742cc1c828 (127.12.234.2:45601), ca00b8cfda2742ff9e79c74f2c62a843 (127.12.234.3:45063)
I20260809 04:47:02.407022 29083 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3acdad44042f4e5888dbe9fc4c066743" candidate_uuid: "a47af5c80dac43b19d033adba1a62d4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a69dc7a46a140f49df77a742cc1c828"
I20260809 04:47:02.407053 29226 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3acdad44042f4e5888dbe9fc4c066743" candidate_uuid: "a47af5c80dac43b19d033adba1a62d4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca00b8cfda2742ff9e79c74f2c62a843"
I20260809 04:47:02.407214 29083 raft_consensus.cc:3037] T 3acdad44042f4e5888dbe9fc4c066743 P 2a69dc7a46a140f49df77a742cc1c828 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:47:02.407259 29226 raft_consensus.cc:3037] T 3acdad44042f4e5888dbe9fc4c066743 P ca00b8cfda2742ff9e79c74f2c62a843 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:47:02.409333 29083 raft_consensus.cc:2454] T 3acdad44042f4e5888dbe9fc4c066743 P 2a69dc7a46a140f49df77a742cc1c828 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a47af5c80dac43b19d033adba1a62d4b in term 1.
I20260809 04:47:02.409395 29226 raft_consensus.cc:2454] T 3acdad44042f4e5888dbe9fc4c066743 P ca00b8cfda2742ff9e79c74f2c62a843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a47af5c80dac43b19d033adba1a62d4b in term 1.
I20260809 04:47:02.409737 28859 leader_election.cc:302] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a69dc7a46a140f49df77a742cc1c828, a47af5c80dac43b19d033adba1a62d4b; no voters:
I20260809 04:47:02.410041 29303 raft_consensus.cc:2781] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:47:02.410485 29303 raft_consensus.cc:686] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [term 1 LEADER]: Becoming Leader. State: Replica: a47af5c80dac43b19d033adba1a62d4b, State: Running, Role: LEADER
I20260809 04:47:02.410926 29303 consensus_queue.cc:237] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69dc7a46a140f49df77a742cc1c828" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45601 } } peers { permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 } } peers { permanent_uuid: "a47af5c80dac43b19d033adba1a62d4b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 44033 } }
I20260809 04:47:02.415138 28769 catalog_manager.cc:5495] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b reported cstate change: term changed from 0 to 1, leader changed from <none> to a47af5c80dac43b19d033adba1a62d4b (127.12.234.1). New cstate: current_term: 1 leader_uuid: "a47af5c80dac43b19d033adba1a62d4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69dc7a46a140f49df77a742cc1c828" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a47af5c80dac43b19d033adba1a62d4b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 44033 } health_report { overall_health: HEALTHY } } }
I20260809 04:47:02.445441 29296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:02.440770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35038 (local address 127.12.234.1:44033)
0809 04:47:02.440947 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:02.440952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:02.440966 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:47:02.441045 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:02.441048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:02.441114 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:02.441229 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:47:02.441549 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.442151 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.443556 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.443963 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.444050 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:02.444503 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:47:02.444997 (+ 494us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:47:02.445092 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:02.445193 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 04:47:02.445249 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 04:47:02.452543 29226 raft_consensus.cc:1261] T 3acdad44042f4e5888dbe9fc4c066743 P ca00b8cfda2742ff9e79c74f2c62a843 [term 1 FOLLOWER]: Refusing update from remote peer a47af5c80dac43b19d033adba1a62d4b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:47:02.452543 29083 raft_consensus.cc:1261] T 3acdad44042f4e5888dbe9fc4c066743 P 2a69dc7a46a140f49df77a742cc1c828 [term 1 FOLLOWER]: Refusing update from remote peer a47af5c80dac43b19d033adba1a62d4b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:47:02.453483 29308 consensus_queue.cc:1035] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:47:02.453841 29303 consensus_queue.cc:1035] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a69dc7a46a140f49df77a742cc1c828" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:47:02.466884 29313 mvcc.cc:205] Tried to move back new op lower bound from 7316483368757350400 to 7316483368598802432. Current Snapshot: MvccSnapshot[applied={T|T < 7316483368757350400}]
I20260809 04:47:02.470381 29314 mvcc.cc:205] Tried to move back new op lower bound from 7316483368757350400 to 7316483368598802432. Current Snapshot: MvccSnapshot[applied={T|T < 7316483368757350400}]
I20260809 04:47:02.478581 28769 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:55404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:47:02.478796 28769 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:55404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:47:02.482046 28769 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca: Sending DeleteTablet for 3 replicas of tablet 3acdad44042f4e5888dbe9fc4c066743
I20260809 04:47:02.483045 29062 tablet_service.cc:1514] Processing DeleteTablet for tablet 3acdad44042f4e5888dbe9fc4c066743 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:47:02 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:57094
I20260809 04:47:02.483074 29206 tablet_service.cc:1514] Processing DeleteTablet for tablet 3acdad44042f4e5888dbe9fc4c066743 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:47:02 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:37526
I20260809 04:47:02.483381 28915 tablet_service.cc:1514] Processing DeleteTablet for tablet 3acdad44042f4e5888dbe9fc4c066743 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:47:02 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:35032
I20260809 04:47:02.483814 29321 tablet_replica.cc:330] T 3acdad44042f4e5888dbe9fc4c066743 P ca00b8cfda2742ff9e79c74f2c62a843: stopping tablet replica
I20260809 04:47:02.484030 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 28814
I20260809 04:47:02.484216 29321 raft_consensus.cc:2229] T 3acdad44042f4e5888dbe9fc4c066743 P ca00b8cfda2742ff9e79c74f2c62a843 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:47:02.484668 29321 raft_consensus.cc:2258] T 3acdad44042f4e5888dbe9fc4c066743 P ca00b8cfda2742ff9e79c74f2c62a843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:47:02.490191 29321 ts_tablet_manager.cc:1905] T 3acdad44042f4e5888dbe9fc4c066743 P ca00b8cfda2742ff9e79c74f2c62a843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:47:02.494185 29322 tablet_replica.cc:330] T 3acdad44042f4e5888dbe9fc4c066743 P 2a69dc7a46a140f49df77a742cc1c828: stopping tablet replica
I20260809 04:47:02.494779 29322 raft_consensus.cc:2229] T 3acdad44042f4e5888dbe9fc4c066743 P 2a69dc7a46a140f49df77a742cc1c828 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:47:02.495329 29322 raft_consensus.cc:2258] T 3acdad44042f4e5888dbe9fc4c066743 P 2a69dc7a46a140f49df77a742cc1c828 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:47:02.497524 29322 ts_tablet_manager.cc:1905] T 3acdad44042f4e5888dbe9fc4c066743 P 2a69dc7a46a140f49df77a742cc1c828: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:47:02.501847 29296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:02.484791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35050 (local address 127.12.234.1:44033)
0809 04:47:02.484945 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:02.484950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:02.485026 (+ 76us) server_negotiation.cc:407] Connection header received
0809 04:47:02.485224 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:02.485228 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:02.485302 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:02.485394 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:02.486817 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.487929 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.489922 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.490559 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.490686 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:02.493989 (+ 3303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:02.494002 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:02.494017 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:02.494063 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:02.497964 (+ 3901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:02.498785 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:02.498793 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:02.498796 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:02.498962 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:02.499375 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:02.499380 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:02.499383 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:02.499765 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:02.500110 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:02.501246 (+ 1136us) server_negotiation.cc:299] Negotiation successful
0809 04:47:02.501483 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 04:47:02.502475 28981 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:47:02.506712 29322 ts_tablet_manager.cc:1918] T 3acdad44042f4e5888dbe9fc4c066743 P 2a69dc7a46a140f49df77a742cc1c828: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:47:02.507649 29322 log.cc:1196] T 3acdad44042f4e5888dbe9fc4c066743 P 2a69dc7a46a140f49df77a742cc1c828: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/wal/wals/3acdad44042f4e5888dbe9fc4c066743
I20260809 04:47:02.508587 29322 ts_tablet_manager.cc:1939] T 3acdad44042f4e5888dbe9fc4c066743 P 2a69dc7a46a140f49df77a742cc1c828: Deleting consensus metadata
I20260809 04:47:02.510886 28751 catalog_manager.cc:4841] TS 2a69dc7a46a140f49df77a742cc1c828 (127.12.234.2:45601): tablet 3acdad44042f4e5888dbe9fc4c066743 (table test-table [id=cdc1f9815083413fb219cd0e93eda55e]) successfully deleted
I20260809 04:47:02.512171 29321 ts_tablet_manager.cc:1918] T 3acdad44042f4e5888dbe9fc4c066743 P ca00b8cfda2742ff9e79c74f2c62a843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:47:02.512347 29321 log.cc:1196] T 3acdad44042f4e5888dbe9fc4c066743 P ca00b8cfda2742ff9e79c74f2c62a843: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/wal/wals/3acdad44042f4e5888dbe9fc4c066743
I20260809 04:47:02.512946 29321 ts_tablet_manager.cc:1939] T 3acdad44042f4e5888dbe9fc4c066743 P ca00b8cfda2742ff9e79c74f2c62a843: Deleting consensus metadata
I20260809 04:47:02.514703 28753 catalog_manager.cc:4841] TS ca00b8cfda2742ff9e79c74f2c62a843 (127.12.234.3:45063): tablet 3acdad44042f4e5888dbe9fc4c066743 (table test-table [id=cdc1f9815083413fb219cd0e93eda55e]) successfully deleted
I20260809 04:47:02.603879 29324 tablet_replica.cc:330] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b: stopping tablet replica
I20260809 04:47:02.604355 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 28814
W20260809 04:47:02.625540 28750 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.234.1:44033 (error 108)
I20260809 04:47:02.625751 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 28991
I20260809 04:47:02.627795 29292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:02.627202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:44033 (local address 127.0.0.1:35066)
0809 04:47:02.627481 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:02.627610 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:44033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
W20260809 04:47:02.628063 28750 catalog_manager.cc:4584] TS a47af5c80dac43b19d033adba1a62d4b (127.12.234.1:44033): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3acdad44042f4e5888dbe9fc4c066743: Network error: Client connection negotiation failed: client connection to 127.12.234.1:44033: connect: Connection refused (error 111)
I20260809 04:47:02.639606 29293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:02.626517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57104 (local address 127.12.234.2:45601)
0809 04:47:02.626769 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:02.626778 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:02.626804 (+ 26us) server_negotiation.cc:407] Connection header received
0809 04:47:02.626879 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:02.626887 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:02.626987 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:02.627108 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:02.628838 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.629605 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.631085 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.631530 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.631633 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:02.634400 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:02.634412 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:02.634426 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:02.634476 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:02.637268 (+ 2792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:02.637873 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:02.637879 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:02.637881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:02.637999 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:02.638296 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:02.638299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:02.638300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:02.638696 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:02.638906 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:02.639093 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 04:47:02.639234 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260809 04:47:02.640594 29128 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:47:02.681730 29292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:02.681239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:44033 (local address 127.0.0.1:35070)
0809 04:47:02.681464 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:02.681570 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:44033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 04:47:02.682760 29292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:02.682381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:44033 (local address 127.0.0.1:35086)
0809 04:47:02.682542 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:02.682635 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:44033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 04:47:02.723304 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 28991
I20260809 04:47:02.743130 29292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:02.742629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:44033 (local address 127.0.0.1:35102)
0809 04:47:02.742836 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:02.742940 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:44033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 04:47:02.744148 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 29138
I20260809 04:47:02.744438 29292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:02.744029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:44033 (local address 127.0.0.1:35108)
0809 04:47:02.744248 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:02.744322 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:44033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260809 04:47:02.756487 29295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:02.744860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37532 (local address 127.12.234.3:45063)
0809 04:47:02.745008 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:02.745012 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:02.745028 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:47:02.745205 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:02.745209 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:02.745266 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:02.745340 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:02.746614 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.747232 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.748446 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:02.748789 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:02.748875 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:02.751541 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:02.751555 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:02.751571 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:02.751599 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:02.754205 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:02.754821 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:02.754828 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:02.754830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:02.754960 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:02.755291 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:02.755294 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:02.755296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:02.755603 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:02.755795 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:02.756063 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 04:47:02.756250 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 04:47:02.757207 29275 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:47:02.825455 29292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:02.824907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:44033 (local address 127.0.0.1:35116)
0809 04:47:02.825160 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:02.825268 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:44033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260809 04:47:02.826620 29292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:02.826140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:44033 (local address 127.0.0.1:35130)
0809 04:47:02.826388 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:02.826471 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:44033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260809 04:47:02.844169 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 29138
I20260809 04:47:02.865877 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 28728
I20260809 04:47:02.866961 28807 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:47:02.969810 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 28728
I20260809 04:47:02.996206 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:44829
--webserver_interface=127.12.234.62
--webserver_port=36611
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:44829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:47:03.305514 29330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:03.305795 29330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:03.305951 29330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:03.312856 29330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:47:03.312954 29330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:03.313056 29330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:03.313144 29330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:47:03.313222 29330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:47:03.322256 29330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:44829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:44829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=36611
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:47:03.323403 29330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:03.324751 29330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:03.336052 29339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:03.336170 29335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:03.336123 29336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:03.336944 29330 server_base.cc:1034] running on GCE node
I20260809 04:47:03.337733 29330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:03.338950 29330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:03.340168 29330 hybrid_clock.cc:648] HybridClock initialized: now 1786250823340128 us; error 53 us; skew 500 ppm
I20260809 04:47:03.340520 29330 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:47:03 dist-test-slave-1jjb krb5kdc[28107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250823, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:03.347308 29330 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:47:03.348802 29330 webserver.cc:460] Webserver started at http://127.12.234.62:36611/ using document root <none> and password file <none>
I20260809 04:47:03.349437 29330 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:03.349539 29330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:03.354540 29330 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:47:03.357180 29347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:03.358417 29330 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:47:03.358547 29330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "03dfd2bb88764bb9b513bf65e731fbca"
format_stamp: "Formatted at 2026-08-09 04:46:57 on dist-test-slave-1jjb"
server_key: "8ac237defb469ddc13cc830fb16d9b56"
server_key_iv: "57fc15cfc39ce527d14eb7a3450eca20"
server_key_version: "encryptionkey@0"
I20260809 04:47:03.359524 29330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:03.379024 29330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:03.382545 29330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:03.382781 29330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:03.400624 29330 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:44829
I20260809 04:47:03.400656 29405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:44829 every 8 connection(s)
I20260809 04:47:03.402046 29330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:47:03.408461 29407 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:47:03.409770 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 29330
I20260809 04:47:03.410815 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:44033
--local_ip_for_outbound_sockets=127.12.234.1
--tserver_master_addrs=127.12.234.62:44829
--webserver_port=40639
--webserver_interface=127.12.234.1
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:47:03.412734 29407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca: Bootstrap starting.
I20260809 04:47:03.435132 29407 log.cc:824] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:03.450304 29407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:47:03.450817 29407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca: Bootstrap complete.
I20260809 04:47:03.459372 29407 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03dfd2bb88764bb9b513bf65e731fbca" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44829 } }
I20260809 04:47:03.460321 29407 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03dfd2bb88764bb9b513bf65e731fbca, State: Initialized, Role: FOLLOWER
I20260809 04:47:03.460894 29407 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03dfd2bb88764bb9b513bf65e731fbca" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44829 } }
I20260809 04:47:03.461143 29407 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:47:03.461274 29407 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:47:03.461478 29407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [term 2 FOLLOWER]: Advancing to term 3
I20260809 04:47:03.465427 29407 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03dfd2bb88764bb9b513bf65e731fbca" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44829 } }
I20260809 04:47:03.465891 29407 leader_election.cc:302] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03dfd2bb88764bb9b513bf65e731fbca; no voters:
I20260809 04:47:03.466562 29407 leader_election.cc:288] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 04:47:03.466715 29411 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [term 3 FOLLOWER]: Leader election won for term 3
I20260809 04:47:03.467897 29411 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [term 3 LEADER]: Becoming Leader. State: Replica: 03dfd2bb88764bb9b513bf65e731fbca, State: Running, Role: LEADER
I20260809 04:47:03.468456 29411 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03dfd2bb88764bb9b513bf65e731fbca" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44829 } }
I20260809 04:47:03.468658 29407 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:47:03.470752 29413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03dfd2bb88764bb9b513bf65e731fbca. Latest consensus state: current_term: 3 leader_uuid: "03dfd2bb88764bb9b513bf65e731fbca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03dfd2bb88764bb9b513bf65e731fbca" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44829 } } }
I20260809 04:47:03.471113 29413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [sys.catalog]: This master's current role is: LEADER
I20260809 04:47:03.471571 29412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "03dfd2bb88764bb9b513bf65e731fbca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03dfd2bb88764bb9b513bf65e731fbca" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44829 } } }
I20260809 04:47:03.471817 29412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03dfd2bb88764bb9b513bf65e731fbca [sys.catalog]: This master's current role is: LEADER
I20260809 04:47:03.472492 29417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:47:03.492833 29417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:47:03.494611 29417 catalog_manager.cc:1259] Loaded cluster ID: 76c454d5ebc84f08a710b36bf9389929
I20260809 04:47:03.494786 29417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 04:47:03.497475 29417 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f7fa18619ad google::LogMessage::Fail() at ??:0
@ 0x7f7fa186488a google::LogMessage::SendToLog() at ??:0
@ 0x7f7fa18614de google::LogMessage::Flush() at ??:0
@ 0x7f7fa1862a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f7fb7294512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f7fb7291976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f7fb7319610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f7fb7319439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f7fa44ad345 kudu::ThreadPool::DispatchThread() at ??:0
I20260809 04:47:03.558508 29433 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f7fa44b091c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f7fa44b0799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f7fa448beaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f7fa77656db start_thread at ??:0
@ 0x7f7fa06fb61f clone at ??:0
W20260809 04:47:03.766417 29409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:03.766700 29409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:03.766861 29409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:03.773873 29409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:03.774022 29409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:03.774192 29409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:47:03.783221 29409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:44033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=40639
--tserver_master_addrs=127.12.234.62:44829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:47:03.784446 29409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:03.785641 29409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:03.798877 29452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:03.798805 29455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:03.798841 29453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:03.799719 29409 server_base.cc:1034] running on GCE node
I20260809 04:47:03.800410 29409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:03.801229 29409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:03.802453 29409 hybrid_clock.cc:648] HybridClock initialized: now 1786250823802415 us; error 35 us; skew 500 ppm
I20260809 04:47:03.802728 29409 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:47:03 dist-test-slave-1jjb krb5kdc[28107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250823, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:03.805881 29409 init.cc:377] Logged in from keytab as kudu/127.12.234.1@KRBTEST.COM (short username kudu)
I20260809 04:47:03.807264 29409 webserver.cc:460] Webserver started at http://127.12.234.1:40639/ using document root <none> and password file <none>
I20260809 04:47:03.807940 29409 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:03.808084 29409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:03.813220 29409 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260809 04:47:03.815963 29464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:03.817282 29409 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:47:03.817512 29409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "a47af5c80dac43b19d033adba1a62d4b"
format_stamp: "Formatted at 2026-08-09 04:46:58 on dist-test-slave-1jjb"
server_key: "8f92676e2861bac6ef2bd56ea15ca845"
server_key_iv: "bc129a89fdc895567f2f6c14cf942498"
server_key_version: "encryptionkey@0"
I20260809 04:47:03.818559 29409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:03.833791 29409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:03.837288 29409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:03.837553 29409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:03.838475 29409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:47:03.840426 29472 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260809 04:47:03.845607 29409 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 04:47:03.845757 29409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260809 04:47:03.845896 29409 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 04:47:03.848898 29409 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 04:47:03.849006 29409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20260809 04:47:03.849330 29472 tablet_bootstrap.cc:492] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b: Bootstrap starting.
I20260809 04:47:03.874872 29472 log.cc:824] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:03.888113 29472 tablet_bootstrap.cc:492] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:47:03.888649 29472 tablet_bootstrap.cc:492] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b: Bootstrap complete.
I20260809 04:47:03.889645 29409 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:44033
I20260809 04:47:03.889748 29600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:44033 every 8 connection(s)
I20260809 04:47:03.890314 29472 ts_tablet_manager.cc:1397] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b: Time spent bootstrapping tablet: real 0.041s user 0.021s sys 0.017s
I20260809 04:47:03.891002 29409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:47:03.892072 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 29409
I20260809 04:47:03.893160 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:45601
--local_ip_for_outbound_sockets=127.12.234.2
--tserver_master_addrs=127.12.234.62:44829
--webserver_port=39859
--webserver_interface=127.12.234.2
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:47:03.897714 29472 raft_consensus.cc:348] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69dc7a46a140f49df77a742cc1c828" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45601 } } peers { permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 } } peers { permanent_uuid: "a47af5c80dac43b19d033adba1a62d4b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 44033 } }
I20260809 04:47:03.898625 29472 raft_consensus.cc:729] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a47af5c80dac43b19d033adba1a62d4b, State: Initialized, Role: FOLLOWER
I20260809 04:47:03.899263 29472 consensus_queue.cc:260] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69dc7a46a140f49df77a742cc1c828" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45601 } } peers { permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 } } peers { permanent_uuid: "a47af5c80dac43b19d033adba1a62d4b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 44033 } }
I20260809 04:47:03.899690 29605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:03.897262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.1:34427)
0809 04:47:03.898228 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:03.898571 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":376,"threads_started":1}
W20260809 04:47:03.900329 29601 heartbeater.cc:644] Failed to heartbeat to 127.12.234.62:44829 (0 consecutive failures): Network error: Failed to ping master at 127.12.234.62:44829: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
I20260809 04:47:03.900969 29472 ts_tablet_manager.cc:1428] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b: Time spent starting tablet: real 0.011s user 0.006s sys 0.005s
I20260809 04:47:03.901271 29605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:03.900857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.1:49723)
0809 04:47:03.901023 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:03.901115 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 04:47:03.902326 29605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:03.901925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.1:36249)
0809 04:47:03.902077 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:03.902172 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 04:47:03.902567 29601 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 04:47:04.215096 29607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:04.215332 29607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:04.215536 29607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:04.222373 29607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:04.222517 29607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:04.222698 29607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:47:04.232481 29607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:45601
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=39859
--tserver_master_addrs=127.12.234.62:44829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:47:04.233613 29607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:04.234826 29607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:04.247258 29612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:04.247296 29615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:04.247219 29613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:04.248273 29607 server_base.cc:1034] running on GCE node
I20260809 04:47:04.248790 29607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:04.249711 29607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:04.250937 29607 hybrid_clock.cc:648] HybridClock initialized: now 1786250824250906 us; error 42 us; skew 500 ppm
I20260809 04:47:04.251228 29607 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:47:04 dist-test-slave-1jjb krb5kdc[28107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250824, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:04.254523 29607 init.cc:377] Logged in from keytab as kudu/127.12.234.2@KRBTEST.COM (short username kudu)
I20260809 04:47:04.255755 29607 webserver.cc:460] Webserver started at http://127.12.234.2:39859/ using document root <none> and password file <none>
I20260809 04:47:04.256361 29607 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:04.256453 29607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:04.261165 29607 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:47:04.263998 29624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:04.265244 29607 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:47:04.265425 29607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "2a69dc7a46a140f49df77a742cc1c828"
format_stamp: "Formatted at 2026-08-09 04:46:59 on dist-test-slave-1jjb"
server_key: "8fe6ece157b8fe538c15637314298f89"
server_key_iv: "f0e39962459b4d96cca01c76fb180c80"
server_key_version: "encryptionkey@0"
I20260809 04:47:04.266264 29607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:04.284888 29607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:04.288378 29607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:04.288626 29607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:04.289611 29607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:47:04.290884 29607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:47:04.290970 29607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:04.291119 29607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:47:04.291198 29607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:04.323911 29607 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:45601
I20260809 04:47:04.323987 29759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:45601 every 8 connection(s)
I20260809 04:47:04.325299 29607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:47:04.330071 29762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:04.328285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.2:56579)
0809 04:47:04.329002 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:04.329258 (+ 256us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":264,"threads_started":1}
W20260809 04:47:04.330790 29760 heartbeater.cc:644] Failed to heartbeat to 127.12.234.62:44829 (0 consecutive failures): Network error: Failed to ping master at 127.12.234.62:44829: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
I20260809 04:47:04.331588 29762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:04.331256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.2:43659)
0809 04:47:04.331404 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:04.331475 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 04:47:04.332412 29762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:04.332035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.2:57071)
0809 04:47:04.332185 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:04.332259 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260809 04:47:04.332665 29760 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 04:47:04.332912 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 29607
I20260809 04:47:04.333542 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:45063
--local_ip_for_outbound_sockets=127.12.234.3
--tserver_master_addrs=127.12.234.62:44829
--webserver_port=44661
--webserver_interface=127.12.234.3
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:47:04.657786 29763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:04.658155 29763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:04.658376 29763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:04.671525 29763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:04.671639 29763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:04.671742 29763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:47:04.680552 29763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:45063
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=44661
--tserver_master_addrs=127.12.234.62:44829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:47:04.681592 29763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:04.682734 29763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:04.695196 29768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:04.695237 29772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:04.695273 29769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:04.696205 29763 server_base.cc:1034] running on GCE node
I20260809 04:47:04.696724 29763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:04.697595 29763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:04.698815 29763 hybrid_clock.cc:648] HybridClock initialized: now 1786250824698778 us; error 46 us; skew 500 ppm
I20260809 04:47:04.699105 29763 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:47:04 dist-test-slave-1jjb krb5kdc[28107](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250824, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:04.702090 29763 init.cc:377] Logged in from keytab as kudu/127.12.234.3@KRBTEST.COM (short username kudu)
I20260809 04:47:04.703421 29763 webserver.cc:460] Webserver started at http://127.12.234.3:44661/ using document root <none> and password file <none>
I20260809 04:47:04.704023 29763 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:04.704182 29763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:04.708801 29763 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 04:47:04.711236 29782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:04.712549 29763 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:47:04.712709 29763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "ca00b8cfda2742ff9e79c74f2c62a843"
format_stamp: "Formatted at 2026-08-09 04:46:59 on dist-test-slave-1jjb"
server_key: "38339b9d93170199d348fe64d6b41f5c"
server_key_iv: "6424c17a5cb5be714b43c3870b615102"
server_key_version: "encryptionkey@0"
I20260809 04:47:04.713616 29763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:04.725051 29763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:04.728266 29763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:04.728502 29763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:04.729388 29763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:47:04.730643 29763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:47:04.730753 29763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:04.730881 29763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:47:04.730974 29763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:04.762073 29763 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:45063
I20260809 04:47:04.762156 29915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:45063 every 8 connection(s)
I20260809 04:47:04.763376 29763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:47:04.767836 29918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:04.766149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.3:47357)
0809 04:47:04.766863 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:04.767081 (+ 218us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":271,"threads_started":1}
I20260809 04:47:04.768528 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 29763
W20260809 04:47:04.768543 29916 heartbeater.cc:644] Failed to heartbeat to 127.12.234.62:44829 (0 consecutive failures): Network error: Failed to ping master at 127.12.234.62:44829: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
I20260809 04:47:04.769423 29918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:04.769004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.3:52665)
0809 04:47:04.769173 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:04.769280 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 04:47:04.770468 29918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:04.770017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.3:55893)
0809 04:47:04.770160 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:04.770271 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 04:47:04.770804 29916 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 04:47:04.903827 29920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:04.903077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.1:46487)
0809 04:47:04.903530 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:04.903655 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":231,"threads_started":1}
I20260809 04:47:05.246534 29922 raft_consensus.cc:482] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:47:05.246918 29922 raft_consensus.cc:504] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69dc7a46a140f49df77a742cc1c828" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45601 } } peers { permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 } } peers { permanent_uuid: "a47af5c80dac43b19d033adba1a62d4b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 44033 } }
I20260809 04:47:05.248625 29922 leader_election.cc:288] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2a69dc7a46a140f49df77a742cc1c828 (127.12.234.2:45601), ca00b8cfda2742ff9e79c74f2c62a843 (127.12.234.3:45063)
Aug 09 04:47:05 dist-test-slave-1jjb krb5kdc[28107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250823, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for kudu/127.12.234.3@KRBTEST.COM
Aug 09 04:47:05 dist-test-slave-1jjb krb5kdc[28107](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250823, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for kudu/127.12.234.2@KRBTEST.COM
I20260809 04:47:05.269972 29920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:05.248996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:45601 (local address 127.12.234.1:42703)
0809 04:47:05.249345 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:05.249389 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:05.250032 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:05.254605 (+ 4573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:05.254612 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:05.255122 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:05.255742 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:05.255759 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:05.257667 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:05.257671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:05.258984 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:05.259000 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:05.260218 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:05.260222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:05.260334 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:05.261090 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:05.261107 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:05.263426 (+ 2319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:05.266473 (+ 3047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:05.266476 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:05.266478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:05.266908 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:05.267411 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:05.267415 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:05.267419 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:05.267636 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:05.268508 (+ 872us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:05.268513 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:05.268727 (+ 214us) client_negotiation.cc:769] Sending connection context
0809 04:47:05.269000 (+ 273us) client_negotiation.cc:240] Negotiation successful
0809 04:47:05.269520 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":135,"spinlock_wait_cycles":17152}
I20260809 04:47:05.270081 29925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:05.249241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:34825 (local address 127.12.234.3:45063)
0809 04:47:05.251199 (+ 1958us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:05.251216 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:05.251270 (+ 54us) server_negotiation.cc:407] Connection header received
0809 04:47:05.253685 (+ 2415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:05.253698 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:05.254157 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:05.254532 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:05.256129 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:05.257460 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:05.259156 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:05.259707 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:05.259831 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:05.263384 (+ 3553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:05.263394 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:05.263409 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:05.263448 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:05.266265 (+ 2817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:05.267000 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:05.267007 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:05.267015 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:05.267189 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:05.267715 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:05.267722 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:05.267728 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:05.268371 (+ 643us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:05.268534 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:05.269249 (+ 715us) server_negotiation.cc:299] Negotiation successful
0809 04:47:05.269681 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1760,"thread_start_us":1621,"threads_started":1}
I20260809 04:47:05.270123 29923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:05.249043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:45063 (local address 127.12.234.1:34825)
0809 04:47:05.250900 (+ 1857us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:05.250922 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:05.251056 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:05.255095 (+ 4039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:05.255098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:05.255592 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:05.255920 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:05.255933 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:05.257668 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:05.257672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:05.258988 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:05.259001 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:05.259904 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:05.259911 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:05.260187 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:05.260904 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:05.260925 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:05.263111 (+ 2186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:05.266389 (+ 3278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:05.266398 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:05.266409 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:05.266878 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:05.267343 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:05.267347 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:05.267350 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:05.267597 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:05.268629 (+ 1032us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:05.268632 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:05.268737 (+ 105us) client_negotiation.cc:769] Sending connection context
0809 04:47:05.269000 (+ 263us) client_negotiation.cc:240] Negotiation successful
0809 04:47:05.269681 (+ 681us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1677,"mutex_wait_us":237,"spinlock_wait_cycles":896,"thread_start_us":1440,"threads_started":1}
I20260809 04:47:05.270738 29924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:05.249219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:42703 (local address 127.12.234.2:45601)
0809 04:47:05.251509 (+ 2290us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:05.251524 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:05.251565 (+ 41us) server_negotiation.cc:407] Connection header received
0809 04:47:05.253701 (+ 2136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:05.253711 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:05.254157 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:05.254532 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:05.255945 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:05.257450 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:05.259131 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:05.260008 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:05.260200 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:05.263661 (+ 3461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:05.263667 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:05.263676 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:05.263711 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:05.266309 (+ 2598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:05.267077 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:05.267085 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:05.267091 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:05.267286 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:05.267771 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:05.267776 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:05.267779 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:05.268362 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:05.268534 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:05.269231 (+ 697us) server_negotiation.cc:299] Negotiation successful
0809 04:47:05.269655 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2107,"thread_start_us":1986,"threads_started":1}
W20260809 04:47:05.273428 29468 leader_election.cc:341] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca00b8cfda2742ff9e79c74f2c62a843 (127.12.234.3:45063): Not found: Tablet not found: 3acdad44042f4e5888dbe9fc4c066743
W20260809 04:47:05.273675 29467 leader_election.cc:341] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a69dc7a46a140f49df77a742cc1c828 (127.12.234.2:45601): Not found: Tablet not found: 3acdad44042f4e5888dbe9fc4c066743
I20260809 04:47:05.273792 29467 leader_election.cc:302] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a47af5c80dac43b19d033adba1a62d4b; no voters: 2a69dc7a46a140f49df77a742cc1c828, ca00b8cfda2742ff9e79c74f2c62a843
I20260809 04:47:05.274106 29922 raft_consensus.cc:2726] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 04:47:05.333950 29926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:05.333127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.2:54171)
0809 04:47:05.333595 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:05.333725 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":250,"threads_started":1}
I20260809 04:47:05.771952 29927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:05.771308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.3:47445)
0809 04:47:05.771702 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:05.771790 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":203,"threads_started":1}
I20260809 04:47:05.905306 29928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:05.904620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.1:48683)
0809 04:47:05.905050 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:05.905133 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":247,"threads_started":1}
I20260809 04:47:06.335449 29929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:06.334846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.2:41393)
0809 04:47:06.335239 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:06.335311 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":214,"threads_started":1}
I20260809 04:47:06.773366 29930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:06.772739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.3:40653)
0809 04:47:06.773123 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:06.773197 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":201,"threads_started":1}
I20260809 04:47:06.906687 29932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:06.906070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.1:34361)
0809 04:47:06.906466 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:06.906535 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":177,"threads_started":1}
I20260809 04:47:07.005363 29933 raft_consensus.cc:482] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:47:07.005606 29933 raft_consensus.cc:504] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69dc7a46a140f49df77a742cc1c828" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45601 } } peers { permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 } } peers { permanent_uuid: "a47af5c80dac43b19d033adba1a62d4b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 44033 } }
I20260809 04:47:07.006503 29933 leader_election.cc:288] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2a69dc7a46a140f49df77a742cc1c828 (127.12.234.2:45601), ca00b8cfda2742ff9e79c74f2c62a843 (127.12.234.3:45063)
W20260809 04:47:07.007442 29467 leader_election.cc:341] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a69dc7a46a140f49df77a742cc1c828 (127.12.234.2:45601): Not found: Tablet not found: 3acdad44042f4e5888dbe9fc4c066743
W20260809 04:47:07.007618 29468 leader_election.cc:341] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca00b8cfda2742ff9e79c74f2c62a843 (127.12.234.3:45063): Not found: Tablet not found: 3acdad44042f4e5888dbe9fc4c066743
I20260809 04:47:07.007767 29468 leader_election.cc:302] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a47af5c80dac43b19d033adba1a62d4b; no voters: 2a69dc7a46a140f49df77a742cc1c828, ca00b8cfda2742ff9e79c74f2c62a843
I20260809 04:47:07.008029 29933 raft_consensus.cc:2726] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 04:47:07.336928 29934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:07.336291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.2:56313)
0809 04:47:07.336697 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:07.336771 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":218,"threads_started":1}
I20260809 04:47:07.774863 29936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:07.774144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.3:38485)
0809 04:47:07.774637 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:07.774715 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":297,"threads_started":1}
I20260809 04:47:07.908283 29937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:07.907594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.1:57097)
0809 04:47:07.908008 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:07.908131 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":212,"threads_started":1}
I20260809 04:47:08.338438 29938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:08.337727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.2:54541)
0809 04:47:08.338134 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:08.338205 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":201,"threads_started":1}
I20260809 04:47:08.776342 29939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:08.775633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.3:49311)
0809 04:47:08.776019 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:08.776149 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":194,"threads_started":1}
I20260809 04:47:08.910102 29940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:08.909336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.1:58481)
0809 04:47:08.909833 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:08.909919 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":241,"threads_started":1}
I20260809 04:47:09.339962 29941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:09.339337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.2:37835)
0809 04:47:09.339732 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:09.339815 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":203,"threads_started":1}
I20260809 04:47:09.777877 29942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:09.777103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.3:34123)
0809 04:47:09.777622 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:09.777708 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":242,"threads_started":1}
I20260809 04:47:09.911672 29943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:09.910978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.1:45019)
0809 04:47:09.911421 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:09.911509 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":231,"threads_started":1}
I20260809 04:47:09.963739 29944 raft_consensus.cc:482] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:47:09.963995 29944 raft_consensus.cc:504] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69dc7a46a140f49df77a742cc1c828" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45601 } } peers { permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 } } peers { permanent_uuid: "a47af5c80dac43b19d033adba1a62d4b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 44033 } }
I20260809 04:47:09.964910 29944 leader_election.cc:288] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2a69dc7a46a140f49df77a742cc1c828 (127.12.234.2:45601), ca00b8cfda2742ff9e79c74f2c62a843 (127.12.234.3:45063)
W20260809 04:47:09.965814 29467 leader_election.cc:341] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a69dc7a46a140f49df77a742cc1c828 (127.12.234.2:45601): Not found: Tablet not found: 3acdad44042f4e5888dbe9fc4c066743
W20260809 04:47:09.965974 29468 leader_election.cc:341] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca00b8cfda2742ff9e79c74f2c62a843 (127.12.234.3:45063): Not found: Tablet not found: 3acdad44042f4e5888dbe9fc4c066743
I20260809 04:47:09.966156 29468 leader_election.cc:302] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a47af5c80dac43b19d033adba1a62d4b; no voters: 2a69dc7a46a140f49df77a742cc1c828, ca00b8cfda2742ff9e79c74f2c62a843
I20260809 04:47:09.966435 29944 raft_consensus.cc:2726] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 04:47:10.341609 29947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:10.340945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.2:46287)
0809 04:47:10.341374 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:10.341449 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":226,"threads_started":1}
I20260809 04:47:10.779472 29948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:10.778738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.3:33687)
0809 04:47:10.779211 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:10.779284 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":256,"threads_started":1}
I20260809 04:47:10.913157 29949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:10.912499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.1:47193)
0809 04:47:10.912899 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:10.912987 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":201,"threads_started":1}
I20260809 04:47:11.343277 29950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:11.342521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.2:39997)
0809 04:47:11.343021 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:11.343104 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":212,"threads_started":1}
I20260809 04:47:11.781206 29951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:11.780455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.3:44607)
0809 04:47:11.780912 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:11.780998 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":205,"threads_started":1}
I20260809 04:47:11.914647 29952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:11.913943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.1:57519)
0809 04:47:11.914402 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:11.914480 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":213,"threads_started":1}
I20260809 04:47:12.344906 29953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:12.344226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.2:56581)
0809 04:47:12.344653 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:12.344727 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":221,"threads_started":1}
I20260809 04:47:12.782711 29955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:12.782044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.3:53235)
0809 04:47:12.782494 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:12.782565 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":234,"threads_started":1}
I20260809 04:47:12.916200 29956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:12.915486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.1:51449)
0809 04:47:12.915885 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:12.915967 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":204,"threads_started":1}
I20260809 04:47:13.346328 29957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:13.345730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.2:56135)
0809 04:47:13.346130 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:13.346199 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":198,"threads_started":1}
I20260809 04:47:13.784395 29958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:13.783576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.3:36671)
0809 04:47:13.784155 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:13.784234 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":244,"threads_started":1}
I20260809 04:47:13.917685 29959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:13.917044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.1:51463)
0809 04:47:13.917446 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:13.917517 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":202,"threads_started":1}
I20260809 04:47:14.347826 29960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:14.347165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.2:55035)
0809 04:47:14.347583 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:14.347657 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":230,"threads_started":1}
I20260809 04:47:14.786154 29962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:14.785350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.3:60343)
0809 04:47:14.785870 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:14.785950 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":257,"threads_started":1}
I20260809 04:47:14.913805 29963 raft_consensus.cc:482] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:47:14.914072 29963 raft_consensus.cc:504] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69dc7a46a140f49df77a742cc1c828" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45601 } } peers { permanent_uuid: "ca00b8cfda2742ff9e79c74f2c62a843" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 45063 } } peers { permanent_uuid: "a47af5c80dac43b19d033adba1a62d4b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 44033 } }
I20260809 04:47:14.914873 29963 leader_election.cc:288] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2a69dc7a46a140f49df77a742cc1c828 (127.12.234.2:45601), ca00b8cfda2742ff9e79c74f2c62a843 (127.12.234.3:45063)
W20260809 04:47:14.915856 29468 leader_election.cc:341] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ca00b8cfda2742ff9e79c74f2c62a843 (127.12.234.3:45063): Not found: Tablet not found: 3acdad44042f4e5888dbe9fc4c066743
W20260809 04:47:14.916060 29467 leader_election.cc:341] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2a69dc7a46a140f49df77a742cc1c828 (127.12.234.2:45601): Not found: Tablet not found: 3acdad44042f4e5888dbe9fc4c066743
I20260809 04:47:14.916265 29467 leader_election.cc:302] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a47af5c80dac43b19d033adba1a62d4b; no voters: 2a69dc7a46a140f49df77a742cc1c828, ca00b8cfda2742ff9e79c74f2c62a843
I20260809 04:47:14.916546 29963 raft_consensus.cc:2726] T 3acdad44042f4e5888dbe9fc4c066743 P a47af5c80dac43b19d033adba1a62d4b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 04:47:14.919286 29964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:14.918482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.1:59619)
0809 04:47:14.918937 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:14.919072 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":223,"threads_started":1}
I20260809 04:47:15.349375 29965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:15.348666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.2:52407)
0809 04:47:15.349147 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:15.349225 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":210,"threads_started":1}
I20260809 04:47:15.787925 29966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:15.787148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.3:50105)
0809 04:47:15.787676 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:15.787775 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":273,"threads_started":1}
I20260809 04:47:15.920832 29967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:15.920206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.1:60231)
0809 04:47:15.920615 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:15.920688 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":235,"threads_started":1}
I20260809 04:47:16.350904 29968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:16.350196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.2:51815)
0809 04:47:16.350672 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:16.350746 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":253,"threads_started":1}
I20260809 04:47:16.789428 29969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:16.788733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.3:46475)
0809 04:47:16.789176 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:16.789249 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":223,"threads_started":1}
I20260809 04:47:16.922480 29970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:16.921722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.1:35247)
0809 04:47:16.922229 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:16.922313 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":305,"threads_started":1}
I20260809 04:47:17.352460 29971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:17.351736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.2:51007)
0809 04:47:17.352208 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:17.352302 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":235,"threads_started":1}
I20260809 04:47:17.790855 29973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:17.790222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.3:47769)
0809 04:47:17.790614 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:17.790713 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":216,"threads_started":1}
I20260809 04:47:17.923921 29975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:17.923210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.1:35933)
0809 04:47:17.923640 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:17.923714 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":216,"threads_started":1}
I20260809 04:47:18.354041 29978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:18.353330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.2:51627)
0809 04:47:18.353760 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:18.353848 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":230,"threads_started":1}
I20260809 04:47:18.792739 29979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:18.791753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.3:34423)
0809 04:47:18.792379 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:18.792480 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":272,"threads_started":1}
I20260809 04:47:18.925472 29980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:18.924854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.1:56919)
0809 04:47:18.925244 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:18.925320 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":198,"threads_started":1}
I20260809 04:47:19.356055 29981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:19.355203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.2:44999)
0809 04:47:19.355808 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:19.355897 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":296,"threads_started":1}
I20260809 04:47:19.777433 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 29409
I20260809 04:47:19.794528 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:19.793853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44829 (local address 127.12.234.3:43089)
0809 04:47:19.794298 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:19.794377 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:44829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":244,"threads_started":1}
I20260809 04:47:19.796686 29982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:19.778460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56448 (local address 127.12.234.1:44033)
0809 04:47:19.779015 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:19.779031 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:19.779059 (+ 28us) server_negotiation.cc:407] Connection header received
0809 04:47:19.779314 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:19.779342 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:19.779643 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:19.779828 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:19.781807 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:19.782811 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:19.784044 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:19.784683 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:19.784778 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:19.790695 (+ 5917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:19.790705 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:19.790718 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:19.790747 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:19.793949 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:19.794600 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:19.794606 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:19.794608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:19.794773 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:19.795176 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:19.795180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:19.795183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:19.795527 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:19.795835 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:19.796166 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 04:47:19.796432 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":876,"threads_started":1}
I20260809 04:47:19.797822 29595 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:47:19.880254 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 29409
I20260809 04:47:19.902144 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 29607
I20260809 04:47:19.915781 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:19.902816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41958 (local address 127.12.234.2:45601)
0809 04:47:19.903352 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:19.903356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:19.903379 (+ 23us) server_negotiation.cc:407] Connection header received
0809 04:47:19.903432 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:19.903435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:19.903492 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:19.903609 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:19.905085 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:19.905839 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:19.907051 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:19.907447 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:19.907577 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:19.910506 (+ 2929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:19.910511 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:19.910515 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:19.910547 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:19.913142 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:19.913829 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:19.913835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:19.913838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:19.913984 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:19.914342 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:19.914347 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:19.914350 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:19.914634 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:19.914887 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:19.915180 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 04:47:19.915417 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":311,"threads_started":1}
I20260809 04:47:19.916738 29755 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:47:19.996433 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 29607
I20260809 04:47:20.017972 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 29763
I20260809 04:47:20.031585 29988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:20.018844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56006 (local address 127.12.234.3:45063)
0809 04:47:20.019396 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:20.019401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:20.019417 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:47:20.019471 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:20.019474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:20.019566 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:20.019695 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:20.021208 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:20.022002 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:20.023187 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:20.023621 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:20.023749 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:20.026368 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:20.026373 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:20.026377 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:20.026407 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:20.028999 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:20.029648 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:20.029655 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:20.029659 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:20.029782 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:20.030170 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:20.030175 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:20.030179 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:20.030438 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:20.030722 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:20.031004 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 04:47:20.031210 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":286,"threads_started":1}
I20260809 04:47:20.032457 29910 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:47:20.109752 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 29763
2026-08-09T04:47:20Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (23820 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:47:20 dist-test-slave-1jjb krb5kdc[29992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:47:20 dist-test-slave-1jjb krb5kdc[29992](info): set up 2 sockets
Aug 09 04:47:20 dist-test-slave-1jjb krb5kdc[29992](info): commencing operation
krb5kdc: starting...
W20260809 04:47:21.203428 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.001s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:47:21 dist-test-slave-1jjb krb5kdc[29992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:47:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:47:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:47:21.441310 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:40967
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:42447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:40967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:47:21.749437 30008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:21.749665 30008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:21.749807 30008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:21.756207 30008 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 04:47:21.756280 30008 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 04:47:21.756347 30008 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 04:47:21.756438 30008 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:47:21.756521 30008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:21.756633 30008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:21.756695 30008 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:47:21.756764 30008 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:47:21.765413 30008 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:42447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:40967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:40967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:47:21.766517 30008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:21.767715 30008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:21.778638 30018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:21.778680 30016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:21.778702 30015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:21.778869 30008 server_base.cc:1034] running on GCE node
I20260809 04:47:21.779834 30008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:21.781037 30008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:21.782279 30008 hybrid_clock.cc:648] HybridClock initialized: now 1786250841782219 us; error 33 us; skew 500 ppm
I20260809 04:47:21.782605 30008 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:47:21 dist-test-slave-1jjb krb5kdc[29992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250841, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:21.785591 30008 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:47:21.787084 30008 webserver.cc:460] Webserver started at http://127.12.234.62:38989/ using document root <none> and password file <none>
I20260809 04:47:21.787729 30008 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:21.787845 30008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:21.788190 30008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:47:21.790874 30008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "5695b9eea47c429d98e81934ebd75f98"
format_stamp: "Formatted at 2026-08-09 04:47:21 on dist-test-slave-1jjb"
server_key: "195c367a48003c7baa84a860dca6a8e5"
server_key_iv: "e11c3bafb2da7448cbdbde76601618b0"
server_key_version: "encryptionkey@0"
I20260809 04:47:21.791579 30008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "5695b9eea47c429d98e81934ebd75f98"
format_stamp: "Formatted at 2026-08-09 04:47:21 on dist-test-slave-1jjb"
server_key: "195c367a48003c7baa84a860dca6a8e5"
server_key_iv: "e11c3bafb2da7448cbdbde76601618b0"
server_key_version: "encryptionkey@0"
I20260809 04:47:21.796191 30008 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260809 04:47:21.799379 30025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:21.800624 30008 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 04:47:21.800801 30008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "5695b9eea47c429d98e81934ebd75f98"
format_stamp: "Formatted at 2026-08-09 04:47:21 on dist-test-slave-1jjb"
server_key: "195c367a48003c7baa84a860dca6a8e5"
server_key_iv: "e11c3bafb2da7448cbdbde76601618b0"
server_key_version: "encryptionkey@0"
I20260809 04:47:21.801002 30008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:21.820982 30008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:21.824772 30008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:21.825007 30008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:21.843070 30008 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:40967
I20260809 04:47:21.843109 30080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:40967 every 8 connection(s)
I20260809 04:47:21.844328 30008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:47:21.847756 30082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:47:21.850874 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 30008
I20260809 04:47:21.851118 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:47:21.852049 13224 external_mini_cluster.cc:1428] Setting key 33761c50622a165180ae824af68c82cf
I20260809 04:47:21.860487 30082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98: Bootstrap starting.
I20260809 04:47:21.864071 30082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98: Neither blocks nor log segments found. Creating new log.
I20260809 04:47:21.865428 30082 log.cc:824] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98: Log is configured to *not* fsync() on all Append() calls
Aug 09 04:47:21 dist-test-slave-1jjb krb5kdc[29992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:47:21.868100 30082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98: No bootstrap required, opened a new log
I20260809 04:47:21.875448 30086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:21.854148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41604 (local address 127.12.234.62:40967)
0809 04:47:21.854930 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:21.854940 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:21.854977 (+ 37us) server_negotiation.cc:407] Connection header received
0809 04:47:21.856513 (+ 1536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:21.856532 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:21.856856 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:21.857150 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:21.858973 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:21.860179 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:21.861891 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:21.862568 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:21.862688 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:21.867473 (+ 4785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:21.867482 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:21.867497 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:21.867541 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:21.871378 (+ 3837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:21.872005 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:21.872013 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:21.872022 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:21.872271 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:21.872617 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:21.872623 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:21.872632 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:21.873260 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:21.873543 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:21.873859 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 04:47:21.874230 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":365,"threads_started":1}
I20260809 04:47:21.878105 30082 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695b9eea47c429d98e81934ebd75f98" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 40967 } }
I20260809 04:47:21.878415 30082 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:47:21.878486 30082 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5695b9eea47c429d98e81934ebd75f98, State: Initialized, Role: FOLLOWER
I20260809 04:47:21.878976 30082 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695b9eea47c429d98e81934ebd75f98" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 40967 } }
I20260809 04:47:21.879172 30082 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:47:21.879318 30082 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:47:21.879488 30082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:47:21.881549 30082 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695b9eea47c429d98e81934ebd75f98" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 40967 } }
I20260809 04:47:21.881982 30082 leader_election.cc:302] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5695b9eea47c429d98e81934ebd75f98; no voters:
I20260809 04:47:21.882464 30082 leader_election.cc:288] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:47:21.882606 30088 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:47:21.883647 30088 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [term 1 LEADER]: Becoming Leader. State: Replica: 5695b9eea47c429d98e81934ebd75f98, State: Running, Role: LEADER
I20260809 04:47:21.884184 30088 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695b9eea47c429d98e81934ebd75f98" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 40967 } }
I20260809 04:47:21.884325 30082 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:47:21.886930 30091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5695b9eea47c429d98e81934ebd75f98. Latest consensus state: current_term: 1 leader_uuid: "5695b9eea47c429d98e81934ebd75f98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695b9eea47c429d98e81934ebd75f98" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 40967 } } }
I20260809 04:47:21.887321 30091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [sys.catalog]: This master's current role is: LEADER
I20260809 04:47:21.887619 30089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5695b9eea47c429d98e81934ebd75f98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695b9eea47c429d98e81934ebd75f98" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 40967 } } }
I20260809 04:47:21.888001 30089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [sys.catalog]: This master's current role is: LEADER
W20260809 04:47:21.894563 30105 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:47:21.894701 30105 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:47:21.894845 30099 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:47:21.896545 30099 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:47:21.904575 30099 catalog_manager.cc:1347] Generated new cluster ID: 011cef416d964908a136759b2817a2ae
I20260809 04:47:21.904683 30099 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:47:21.925455 30099 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:47:21.926710 30099 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:47:21.931174 30099 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98: Generated new TSK 0
I20260809 04:47:21.931968 30099 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
I20260809 04:47:21.999009 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:0
--local_ip_for_outbound_sockets=127.12.234.1
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:40967
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:42447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:47:22.317029 30112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:22.317277 30112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:22.317415 30112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:22.324496 30112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:22.324599 30112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:22.324772 30112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:47:22.333745 30112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:42447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:40967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:47:22.334833 30112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:22.336061 30112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:22.348239 30120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:22.348248 30117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:22.348286 30118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:22.348942 30112 server_base.cc:1034] running on GCE node
I20260809 04:47:22.349568 30112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:22.350457 30112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:22.351665 30112 hybrid_clock.cc:648] HybridClock initialized: now 1786250842351620 us; error 51 us; skew 500 ppm
I20260809 04:47:22.351972 30112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:47:22 dist-test-slave-1jjb krb5kdc[29992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250842, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:22.355069 30112 init.cc:377] Logged in from keytab as kudu/127.12.234.1@KRBTEST.COM (short username kudu)
I20260809 04:47:22.356621 30112 webserver.cc:460] Webserver started at http://127.12.234.1:40791/ using document root <none> and password file <none>
I20260809 04:47:22.357229 30112 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:22.357368 30112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:22.357651 30112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:47:22.360476 30112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/data/instance:
uuid: "64c3ecf36c9a436b8893781f5130a8cb"
format_stamp: "Formatted at 2026-08-09 04:47:22 on dist-test-slave-1jjb"
server_key: "553137d64b696ca9eb85cd1d45699e12"
server_key_iv: "3937e74f3c5c6686ec9611523cbe2752"
server_key_version: "encryptionkey@0"
I20260809 04:47:22.361253 30112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance:
uuid: "64c3ecf36c9a436b8893781f5130a8cb"
format_stamp: "Formatted at 2026-08-09 04:47:22 on dist-test-slave-1jjb"
server_key: "553137d64b696ca9eb85cd1d45699e12"
server_key_iv: "3937e74f3c5c6686ec9611523cbe2752"
server_key_version: "encryptionkey@0"
I20260809 04:47:22.366122 30112 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:47:22.369453 30128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:22.370694 30112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 04:47:22.370875 30112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "64c3ecf36c9a436b8893781f5130a8cb"
format_stamp: "Formatted at 2026-08-09 04:47:22 on dist-test-slave-1jjb"
server_key: "553137d64b696ca9eb85cd1d45699e12"
server_key_iv: "3937e74f3c5c6686ec9611523cbe2752"
server_key_version: "encryptionkey@0"
I20260809 04:47:22.371062 30112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:22.389262 30112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:22.392831 30112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:22.393059 30112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:22.393863 30112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:47:22.394985 30112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:47:22.395051 30112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:22.395123 30112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:47:22.395164 30112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:22.426582 30112 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:34001
I20260809 04:47:22.426654 30257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:34001 every 8 connection(s)
I20260809 04:47:22.428248 30112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:47:22.431747 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 30112
I20260809 04:47:22.431967 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance
I20260809 04:47:22.432729 13224 external_mini_cluster.cc:1428] Setting key 7f1b1dfc61434683c1afe7376f43b438
Aug 09 04:47:22 dist-test-slave-1jjb krb5kdc[29992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250842, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:47:22.447472 30261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:22.430926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:41191 (local address 127.12.234.62:40967)
0809 04:47:22.431384 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:22.431402 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:22.432415 (+ 1013us) server_negotiation.cc:407] Connection header received
0809 04:47:22.434513 (+ 2098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:22.434518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:22.434604 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:22.434715 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:22.436109 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:22.436831 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:22.438331 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:22.438697 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:22.438818 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:22.442009 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:22.442015 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:22.442020 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:22.442061 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:22.444614 (+ 2553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:22.445297 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:22.445302 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:22.445306 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:22.445436 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:22.445859 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:22.445864 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:22.445867 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:22.446126 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:22.446282 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:22.446907 (+ 625us) server_negotiation.cc:299] Negotiation successful
0809 04:47:22.447128 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":225,"threads_started":1}
I20260809 04:47:22.448364 30262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:22.431330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.1:41191)
0809 04:47:22.432220 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:22.432271 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:22.434243 (+ 1972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:22.434915 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:22.434922 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:22.435378 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:22.435915 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:22.435938 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:22.437021 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:22.437028 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:22.438157 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:22.438173 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:22.438900 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:22.438907 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:22.439104 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:22.439699 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:22.439722 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:22.441776 (+ 2054us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:22.444750 (+ 2974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:22.444757 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:22.444769 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:22.445171 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:22.445556 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:22.445561 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:22.445564 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:22.445739 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:22.446267 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:22.446274 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:22.446549 (+ 275us) client_negotiation.cc:769] Sending connection context
0809 04:47:22.446814 (+ 265us) client_negotiation.cc:240] Negotiation successful
0809 04:47:22.447260 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":321,"threads_started":1}
I20260809 04:47:22.449798 30259 heartbeater.cc:347] Connected to a master server at 127.12.234.62:40967
I20260809 04:47:22.450073 30259 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:22.450692 30259 heartbeater.cc:511] Master 127.12.234.62:40967 requested a full tablet report, sending...
I20260809 04:47:22.452945 30044 ts_manager.cc:194] Registered new tserver with Master: 64c3ecf36c9a436b8893781f5130a8cb (127.12.234.1:34001)
I20260809 04:47:22.454931 30044 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.1@KRBTEST.COM'} at 127.12.234.1:41191
WARNING: no policy specified for kudu/127.12.234.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
I20260809 04:47:22.491566 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:0
--local_ip_for_outbound_sockets=127.12.234.2
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:40967
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:42447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:47:22.797976 30265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:22.798202 30265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:22.798281 30265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:22.804906 30265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:22.804996 30265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:22.805097 30265 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:47:22.814177 30265 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:42447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:40967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:47:22.815241 30265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:22.816555 30265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:22.829277 30271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:22.829540 30275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:22.829777 30270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:22.830281 30265 server_base.cc:1034] running on GCE node
I20260809 04:47:22.830755 30265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:22.831511 30265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:22.832737 30265 hybrid_clock.cc:648] HybridClock initialized: now 1786250842832681 us; error 47 us; skew 500 ppm
I20260809 04:47:22.832996 30265 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:47:22 dist-test-slave-1jjb krb5kdc[29992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250842, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:22.835789 30265 init.cc:377] Logged in from keytab as kudu/127.12.234.2@KRBTEST.COM (short username kudu)
I20260809 04:47:22.837143 30265 webserver.cc:460] Webserver started at http://127.12.234.2:34997/ using document root <none> and password file <none>
I20260809 04:47:22.837742 30265 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:22.837869 30265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:22.838173 30265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:47:22.840778 30265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/data/instance:
uuid: "a4f0bfd4458e4cbd829e905adf720366"
format_stamp: "Formatted at 2026-08-09 04:47:22 on dist-test-slave-1jjb"
server_key: "9261fad4449bd10e24901bcb0a3b1955"
server_key_iv: "10ca47b25d37ebc037d9d5468c7dc75e"
server_key_version: "encryptionkey@0"
I20260809 04:47:22.841511 30265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance:
uuid: "a4f0bfd4458e4cbd829e905adf720366"
format_stamp: "Formatted at 2026-08-09 04:47:22 on dist-test-slave-1jjb"
server_key: "9261fad4449bd10e24901bcb0a3b1955"
server_key_iv: "10ca47b25d37ebc037d9d5468c7dc75e"
server_key_version: "encryptionkey@0"
I20260809 04:47:22.846647 30265 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 04:47:22.850054 30283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:22.851361 30265 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:47:22.851508 30265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "a4f0bfd4458e4cbd829e905adf720366"
format_stamp: "Formatted at 2026-08-09 04:47:22 on dist-test-slave-1jjb"
server_key: "9261fad4449bd10e24901bcb0a3b1955"
server_key_iv: "10ca47b25d37ebc037d9d5468c7dc75e"
server_key_version: "encryptionkey@0"
I20260809 04:47:22.851714 30265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:22.862429 30265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:22.865782 30265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:22.866019 30265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:22.866791 30265 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:47:22.867921 30265 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:47:22.867990 30265 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:22.868063 30265 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:47:22.868173 30265 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260809 04:47:22.901434 30265 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:36001
I20260809 04:47:22.901533 30412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:36001 every 8 connection(s)
I20260809 04:47:22.902601 30265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:47:22.902870 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 30265
I20260809 04:47:22.903065 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance
I20260809 04:47:22.903851 13224 external_mini_cluster.cc:1428] Setting key b84bd0fe6eb1fb240eba31e12011337f
Aug 09 04:47:22 dist-test-slave-1jjb krb5kdc[29992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250842, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:47:22.924160 30261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:22.905217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:41389 (local address 127.12.234.62:40967)
0809 04:47:22.905397 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:22.905402 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:22.906721 (+ 1319us) server_negotiation.cc:407] Connection header received
0809 04:47:22.908654 (+ 1933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:22.908660 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:22.908765 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:22.908912 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:22.910363 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:22.911117 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:22.912882 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:22.913325 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:22.913472 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:22.917186 (+ 3714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:22.917191 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:22.917194 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:22.917225 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:22.920832 (+ 3607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:22.921517 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:22.921526 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:22.921532 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:22.921699 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:22.922146 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:22.922154 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:22.922160 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:22.922599 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:22.922788 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:22.923392 (+ 604us) server_negotiation.cc:299] Negotiation successful
0809 04:47:22.923664 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 04:47:22.924597 30415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:22.905788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.2:41389)
0809 04:47:22.906572 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:22.906609 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:22.908381 (+ 1772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:22.909142 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:22.909150 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:22.909645 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:22.910170 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:22.910187 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:22.911333 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:22.911337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:22.912669 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:22.912689 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:22.913559 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:22.913567 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:22.913812 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:22.914545 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:22.914569 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:22.916966 (+ 2397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:22.920972 (+ 4006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:22.920977 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:22.920986 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:22.921356 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:22.921810 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:22.921813 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:22.921815 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:22.921983 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:22.922728 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:22.922733 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:22.923001 (+ 268us) client_negotiation.cc:769] Sending connection context
0809 04:47:22.923217 (+ 216us) client_negotiation.cc:240] Negotiation successful
0809 04:47:22.923537 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":305,"threads_started":1}
I20260809 04:47:22.925993 30413 heartbeater.cc:347] Connected to a master server at 127.12.234.62:40967
I20260809 04:47:22.926381 30413 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:22.927188 30413 heartbeater.cc:511] Master 127.12.234.62:40967 requested a full tablet report, sending...
I20260809 04:47:22.928802 30044 ts_manager.cc:194] Registered new tserver with Master: a4f0bfd4458e4cbd829e905adf720366 (127.12.234.2:36001)
I20260809 04:47:22.929703 30044 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.2@KRBTEST.COM'} at 127.12.234.2:41389
WARNING: no policy specified for kudu/127.12.234.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
I20260809 04:47:22.969467 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:0
--local_ip_for_outbound_sockets=127.12.234.3
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:40967
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:42447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:47:23.283473 30419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:23.283738 30419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:23.283891 30419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:23.290809 30419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:23.290948 30419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:23.291105 30419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:47:23.300033 30419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:42447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:40967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:47:23.301257 30419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:23.302549 30419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:23.315073 30424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:23.315114 30425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:23.315241 30427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:23.316375 30419 server_base.cc:1034] running on GCE node
I20260809 04:47:23.316916 30419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:23.317754 30419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:23.318981 30419 hybrid_clock.cc:648] HybridClock initialized: now 1786250843318945 us; error 67 us; skew 500 ppm
I20260809 04:47:23.319262 30419 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:47:23 dist-test-slave-1jjb krb5kdc[29992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250843, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:23.322341 30419 init.cc:377] Logged in from keytab as kudu/127.12.234.3@KRBTEST.COM (short username kudu)
I20260809 04:47:23.323699 30419 webserver.cc:460] Webserver started at http://127.12.234.3:33145/ using document root <none> and password file <none>
I20260809 04:47:23.324424 30419 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:23.324579 30419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:23.324887 30419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:47:23.327564 30419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/data/instance:
uuid: "f2de2489efa24bc591f6a2963c41d6f8"
format_stamp: "Formatted at 2026-08-09 04:47:23 on dist-test-slave-1jjb"
server_key: "fc7dca100bf153b7bad0f2f4a5ab64da"
server_key_iv: "2edc1153d99e56082098e456cbe1c05a"
server_key_version: "encryptionkey@0"
I20260809 04:47:23.328359 30419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance:
uuid: "f2de2489efa24bc591f6a2963c41d6f8"
format_stamp: "Formatted at 2026-08-09 04:47:23 on dist-test-slave-1jjb"
server_key: "fc7dca100bf153b7bad0f2f4a5ab64da"
server_key_iv: "2edc1153d99e56082098e456cbe1c05a"
server_key_version: "encryptionkey@0"
I20260809 04:47:23.333357 30419 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 04:47:23.336529 30434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:23.337667 30419 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260809 04:47:23.337833 30419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "f2de2489efa24bc591f6a2963c41d6f8"
format_stamp: "Formatted at 2026-08-09 04:47:23 on dist-test-slave-1jjb"
server_key: "fc7dca100bf153b7bad0f2f4a5ab64da"
server_key_iv: "2edc1153d99e56082098e456cbe1c05a"
server_key_version: "encryptionkey@0"
I20260809 04:47:23.338018 30419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:23.349002 30419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:23.352734 30419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:23.353011 30419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:23.353860 30419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:47:23.355260 30419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:47:23.355355 30419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:23.355486 30419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:47:23.355607 30419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:23.387732 30419 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:36117
I20260809 04:47:23.387774 30563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:36117 every 8 connection(s)
I20260809 04:47:23.388968 30419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:47:23.393249 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 30419
I20260809 04:47:23.393451 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance
I20260809 04:47:23.393924 13224 external_mini_cluster.cc:1428] Setting key d657e03a21db799d90fad8de8f814ef0
Aug 09 04:47:23 dist-test-slave-1jjb krb5kdc[29992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250843, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:47:23.408208 30261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:23.391446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:44181 (local address 127.12.234.62:40967)
0809 04:47:23.391658 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:23.391664 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:23.392795 (+ 1131us) server_negotiation.cc:407] Connection header received
0809 04:47:23.394722 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:23.394728 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:23.394808 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:23.394935 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:23.396605 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.397541 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.398959 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.399344 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.399435 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:23.402515 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:23.402519 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:23.402522 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:23.402552 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:23.405319 (+ 2767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:23.405991 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:23.405995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:23.405997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:23.406100 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:23.406494 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:23.406498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:23.406501 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:23.406773 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:23.406943 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:23.407583 (+ 640us) server_negotiation.cc:299] Negotiation successful
0809 04:47:23.407775 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 04:47:23.409081 30566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:23.391914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.3:44181)
0809 04:47:23.392637 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:23.392681 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:23.394475 (+ 1794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:23.395198 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:23.395204 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:23.395652 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:23.396424 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:23.396450 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.397702 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.397706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:23.398811 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:23.398827 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.399515 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.399519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:23.399686 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:23.400302 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:23.400331 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:23.402308 (+ 1977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:23.405446 (+ 3138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:23.405454 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:23.405466 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:23.405859 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:23.406206 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:23.406210 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:23.406214 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:23.406382 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:23.406917 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:23.406924 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:23.407209 (+ 285us) client_negotiation.cc:769] Sending connection context
0809 04:47:23.407443 (+ 234us) client_negotiation.cc:240] Negotiation successful
0809 04:47:23.407865 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":319,"threads_started":1}
I20260809 04:47:23.410254 30564 heartbeater.cc:347] Connected to a master server at 127.12.234.62:40967
I20260809 04:47:23.410502 30564 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:23.411023 30564 heartbeater.cc:511] Master 127.12.234.62:40967 requested a full tablet report, sending...
I20260809 04:47:23.412278 30044 ts_manager.cc:194] Registered new tserver with Master: f2de2489efa24bc591f6a2963c41d6f8 (127.12.234.3:36117)
I20260809 04:47:23.413019 30044 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.3@KRBTEST.COM'} at 127.12.234.3:44181
I20260809 04:47:23.421000 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:47:23.438740 30261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:23.425203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41618 (local address 127.12.234.62:40967)
0809 04:47:23.425387 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:23.425410 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:23.425816 (+ 406us) server_negotiation.cc:407] Connection header received
0809 04:47:23.426014 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:23.426019 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:23.426102 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:23.426243 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:23.428271 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.428918 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.431076 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.431425 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.431509 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:23.434184 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:23.434187 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:23.434190 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:23.434214 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:23.436600 (+ 2386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:23.437227 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:23.437230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:23.437232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:23.437338 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:23.437646 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:23.437649 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:23.437650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:23.437940 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:23.438104 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:23.438380 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 04:47:23.438526 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:47:23.442435 30044 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41618:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:47:23.448534 30044 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:47:23.457638 30259 heartbeater.cc:503] Master 127.12.234.62:40967 was elected leader, sending a full tablet report...
I20260809 04:47:23.474807 30578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:23.460233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:36117 (local address 127.0.0.1:44110)
0809 04:47:23.461202 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:23.461260 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:23.461464 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:23.465325 (+ 3861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:23.465349 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:23.465387 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:23.465890 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:23.465903 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.468561 (+ 2658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.468570 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:23.471621 (+ 3051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:23.471648 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.473818 (+ 2170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.473825 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:23.474137 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:23.474195 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 04:47:23.474303 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 04:47:23.474473 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"thread_start_us":8093,"threads_started":1}
I20260809 04:47:23.475939 30579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:23.460709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44110 (local address 127.12.234.3:36117)
0809 04:47:23.464634 (+ 3925us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:23.464645 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:23.464682 (+ 37us) server_negotiation.cc:407] Connection header received
0809 04:47:23.464787 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:23.464796 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:23.465056 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:23.466125 (+ 1069us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:23.466498 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.468426 (+ 1928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.471843 (+ 3417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.473561 (+ 1718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.475096 (+ 1535us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:23.475391 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:23.475462 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 04:47:23.475616 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3734,"thread_start_us":3540,"threads_started":1}
I20260809 04:47:23.480309 30491 tablet_service.cc:1467] Processing CreateTablet for tablet a642aecc16f2495a9ca8bac5a93690fa (DEFAULT_TABLE table=test-table [id=825dac4bf1804affa9649e95cfbf09b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:47:23.480854 30577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:23.461298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:34001 (local address 127.0.0.1:34026)
0809 04:47:23.471953 (+ 10655us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:23.471990 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:23.472243 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:23.473282 (+ 1039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:23.473289 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:23.473319 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:23.473758 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:23.473773 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.475903 (+ 2130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.475907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:23.477749 (+ 1842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:23.477764 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.480108 (+ 2344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.480114 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:23.480276 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:23.480316 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 04:47:23.480423 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 04:47:23.480555 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7189,"thread_start_us":1812,"threads_started":1}
I20260809 04:47:23.480916 30583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:23.461868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34026 (local address 127.12.234.1:34001)
0809 04:47:23.462665 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:23.462677 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:23.472098 (+ 9421us) server_negotiation.cc:407] Connection header received
0809 04:47:23.472460 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:23.472471 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:23.472734 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:23.472913 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:23.473962 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.475738 (+ 1776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.478215 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.479883 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.480018 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:23.480261 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:23.480468 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 04:47:23.480622 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":926,"threads_started":1}
I20260809 04:47:23.481941 30491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a642aecc16f2495a9ca8bac5a93690fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:47:23.483991 30582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:23.460530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:36001 (local address 127.0.0.1:36502)
0809 04:47:23.463398 (+ 2868us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:23.463418 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:23.463537 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:23.476610 (+ 13073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:23.476615 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:23.476643 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:23.476994 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:23.477004 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.479907 (+ 2903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.479912 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:23.481782 (+ 1870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:23.481795 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.483502 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.483507 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:23.483655 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:23.483676 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 04:47:23.483723 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 04:47:23.483796 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2710,"thread_start_us":249,"threads_started":1}
I20260809 04:47:23.484233 30180 tablet_service.cc:1467] Processing CreateTablet for tablet a642aecc16f2495a9ca8bac5a93690fa (DEFAULT_TABLE table=test-table [id=825dac4bf1804affa9649e95cfbf09b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:47:23.484944 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:23.460820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36502 (local address 127.12.234.2:36001)
0809 04:47:23.475906 (+ 15086us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:23.475914 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:23.475938 (+ 24us) server_negotiation.cc:407] Connection header received
0809 04:47:23.476035 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:23.476043 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:23.476253 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:23.477191 (+ 938us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:23.477385 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.479760 (+ 2375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.481943 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.483256 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.484284 (+ 1028us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:23.484493 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:23.484542 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 04:47:23.484646 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14912,"thread_start_us":15546,"threads_started":1}
I20260809 04:47:23.485481 30180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a642aecc16f2495a9ca8bac5a93690fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:47:23.491200 30338 tablet_service.cc:1467] Processing CreateTablet for tablet a642aecc16f2495a9ca8bac5a93690fa (DEFAULT_TABLE table=test-table [id=825dac4bf1804affa9649e95cfbf09b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:47:23.492429 30338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a642aecc16f2495a9ca8bac5a93690fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:47:23.496033 30584 tablet_bootstrap.cc:492] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8: Bootstrap starting.
I20260809 04:47:23.499697 30584 tablet_bootstrap.cc:654] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8: Neither blocks nor log segments found. Creating new log.
I20260809 04:47:23.500867 30584 log.cc:824] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:23.501518 30585 tablet_bootstrap.cc:492] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb: Bootstrap starting.
I20260809 04:47:23.503489 30588 tablet_bootstrap.cc:492] T a642aecc16f2495a9ca8bac5a93690fa P a4f0bfd4458e4cbd829e905adf720366: Bootstrap starting.
I20260809 04:47:23.505084 30585 tablet_bootstrap.cc:654] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb: Neither blocks nor log segments found. Creating new log.
I20260809 04:47:23.505196 30584 tablet_bootstrap.cc:492] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8: No bootstrap required, opened a new log
I20260809 04:47:23.505447 30584 ts_tablet_manager.cc:1397] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260809 04:47:23.506074 30585 log.cc:824] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:23.506865 30588 tablet_bootstrap.cc:654] T a642aecc16f2495a9ca8bac5a93690fa P a4f0bfd4458e4cbd829e905adf720366: Neither blocks nor log segments found. Creating new log.
I20260809 04:47:23.508114 30588 log.cc:824] T a642aecc16f2495a9ca8bac5a93690fa P a4f0bfd4458e4cbd829e905adf720366: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:23.508684 30585 tablet_bootstrap.cc:492] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb: No bootstrap required, opened a new log
I20260809 04:47:23.509034 30585 ts_tablet_manager.cc:1397] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 04:47:23.511191 30588 tablet_bootstrap.cc:492] T a642aecc16f2495a9ca8bac5a93690fa P a4f0bfd4458e4cbd829e905adf720366: No bootstrap required, opened a new log
I20260809 04:47:23.511454 30588 ts_tablet_manager.cc:1397] T a642aecc16f2495a9ca8bac5a93690fa P a4f0bfd4458e4cbd829e905adf720366: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 04:47:23.515890 30584 raft_consensus.cc:348] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2de2489efa24bc591f6a2963c41d6f8" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 36117 } } peers { permanent_uuid: "a4f0bfd4458e4cbd829e905adf720366" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 36001 } } peers { permanent_uuid: "64c3ecf36c9a436b8893781f5130a8cb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 34001 } }
I20260809 04:47:23.516211 30585 raft_consensus.cc:348] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2de2489efa24bc591f6a2963c41d6f8" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 36117 } } peers { permanent_uuid: "a4f0bfd4458e4cbd829e905adf720366" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 36001 } } peers { permanent_uuid: "64c3ecf36c9a436b8893781f5130a8cb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 34001 } }
I20260809 04:47:23.516512 30584 raft_consensus.cc:374] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:47:23.516618 30585 raft_consensus.cc:374] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:47:23.516631 30584 raft_consensus.cc:729] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2de2489efa24bc591f6a2963c41d6f8, State: Initialized, Role: FOLLOWER
I20260809 04:47:23.516741 30585 raft_consensus.cc:729] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64c3ecf36c9a436b8893781f5130a8cb, State: Initialized, Role: FOLLOWER
I20260809 04:47:23.517360 30585 consensus_queue.cc:260] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2de2489efa24bc591f6a2963c41d6f8" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 36117 } } peers { permanent_uuid: "a4f0bfd4458e4cbd829e905adf720366" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 36001 } } peers { permanent_uuid: "64c3ecf36c9a436b8893781f5130a8cb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 34001 } }
I20260809 04:47:23.517287 30584 consensus_queue.cc:260] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2de2489efa24bc591f6a2963c41d6f8" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 36117 } } peers { permanent_uuid: "a4f0bfd4458e4cbd829e905adf720366" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 36001 } } peers { permanent_uuid: "64c3ecf36c9a436b8893781f5130a8cb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 34001 } }
I20260809 04:47:23.519966 30585 ts_tablet_manager.cc:1428] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb: Time spent starting tablet: real 0.011s user 0.005s sys 0.004s
I20260809 04:47:23.520869 30564 heartbeater.cc:503] Master 127.12.234.62:40967 was elected leader, sending a full tablet report...
I20260809 04:47:23.521503 30584 ts_tablet_manager.cc:1428] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8: Time spent starting tablet: real 0.016s user 0.005s sys 0.010s
I20260809 04:47:23.522358 30588 raft_consensus.cc:348] T a642aecc16f2495a9ca8bac5a93690fa P a4f0bfd4458e4cbd829e905adf720366 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2de2489efa24bc591f6a2963c41d6f8" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 36117 } } peers { permanent_uuid: "a4f0bfd4458e4cbd829e905adf720366" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 36001 } } peers { permanent_uuid: "64c3ecf36c9a436b8893781f5130a8cb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 34001 } }
I20260809 04:47:23.522861 30588 raft_consensus.cc:374] T a642aecc16f2495a9ca8bac5a93690fa P a4f0bfd4458e4cbd829e905adf720366 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:47:23.522987 30588 raft_consensus.cc:729] T a642aecc16f2495a9ca8bac5a93690fa P a4f0bfd4458e4cbd829e905adf720366 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4f0bfd4458e4cbd829e905adf720366, State: Initialized, Role: FOLLOWER
I20260809 04:47:23.523638 30588 consensus_queue.cc:260] T a642aecc16f2495a9ca8bac5a93690fa P a4f0bfd4458e4cbd829e905adf720366 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2de2489efa24bc591f6a2963c41d6f8" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 36117 } } peers { permanent_uuid: "a4f0bfd4458e4cbd829e905adf720366" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 36001 } } peers { permanent_uuid: "64c3ecf36c9a436b8893781f5130a8cb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 34001 } }
I20260809 04:47:23.524958 30413 heartbeater.cc:503] Master 127.12.234.62:40967 was elected leader, sending a full tablet report...
I20260809 04:47:23.525365 30588 ts_tablet_manager.cc:1428] T a642aecc16f2495a9ca8bac5a93690fa P a4f0bfd4458e4cbd829e905adf720366: Time spent starting tablet: real 0.014s user 0.009s sys 0.005s
W20260809 04:47:23.640545 30565 tablet.cc:2369] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:47:23.654886 30414 tablet.cc:2369] T a642aecc16f2495a9ca8bac5a93690fa P a4f0bfd4458e4cbd829e905adf720366: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:47:23.680758 30260 tablet.cc:2369] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:47:23.730077 30592 raft_consensus.cc:482] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:47:23.730414 30592 raft_consensus.cc:504] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2de2489efa24bc591f6a2963c41d6f8" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 36117 } } peers { permanent_uuid: "a4f0bfd4458e4cbd829e905adf720366" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 36001 } } peers { permanent_uuid: "64c3ecf36c9a436b8893781f5130a8cb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 34001 } }
I20260809 04:47:23.731999 30592 leader_election.cc:288] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4f0bfd4458e4cbd829e905adf720366 (127.12.234.2:36001), 64c3ecf36c9a436b8893781f5130a8cb (127.12.234.1:34001)
I20260809 04:47:23.738286 30566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:23.732294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:34001 (local address 127.12.234.3:43345)
0809 04:47:23.732508 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:23.732529 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:23.732656 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:23.733291 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:23.733298 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:23.733348 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:23.733541 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:23.733554 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.734776 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.734781 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:23.736285 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:23.736297 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.737553 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.737559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:23.737745 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:23.737783 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 04:47:23.737896 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 04:47:23.738035 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 04:47:23.738300 30583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:23.732691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:43345 (local address 127.12.234.1:34001)
0809 04:47:23.732926 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:23.732931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:23.732953 (+ 22us) server_negotiation.cc:407] Connection header received
0809 04:47:23.733036 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:23.733041 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:23.733122 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:23.733231 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:23.733693 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.734650 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.736455 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.737409 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.737524 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:23.737574 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:23.737923 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 04:47:23.738044 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 04:47:23.738919 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:23.732356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:46257 (local address 127.12.234.2:36001)
0809 04:47:23.732570 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:23.732578 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:23.733247 (+ 669us) server_negotiation.cc:407] Connection header received
0809 04:47:23.733441 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:23.733446 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:23.733522 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:23.733619 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:23.734256 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.735089 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.737015 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.737931 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.738055 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:23.738114 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:23.738529 (+ 415us) server_negotiation.cc:299] Negotiation successful
0809 04:47:23.738686 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:47:23.739467 30594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:23.732297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:36001 (local address 127.12.234.3:46257)
0809 04:47:23.733120 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:23.733143 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:23.733264 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:23.733665 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:23.733669 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:23.733693 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:23.734068 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:23.734079 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.735257 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.735261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:23.736846 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:23.736861 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.738119 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.738125 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:23.738294 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:23.738364 (+ 70us) client_negotiation.cc:769] Sending connection context
0809 04:47:23.739109 (+ 745us) client_negotiation.cc:240] Negotiation successful
0809 04:47:23.739209 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"mutex_wait_us":164,"thread_start_us":329,"threads_started":1}
I20260809 04:47:23.739392 30203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a642aecc16f2495a9ca8bac5a93690fa" candidate_uuid: "f2de2489efa24bc591f6a2963c41d6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64c3ecf36c9a436b8893781f5130a8cb" is_pre_election: true
I20260809 04:47:23.739930 30203 raft_consensus.cc:2454] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2de2489efa24bc591f6a2963c41d6f8 in term 0.
I20260809 04:47:23.740434 30361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a642aecc16f2495a9ca8bac5a93690fa" candidate_uuid: "f2de2489efa24bc591f6a2963c41d6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4f0bfd4458e4cbd829e905adf720366" is_pre_election: true
I20260809 04:47:23.740569 30439 leader_election.cc:302] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64c3ecf36c9a436b8893781f5130a8cb, f2de2489efa24bc591f6a2963c41d6f8; no voters:
I20260809 04:47:23.740926 30592 raft_consensus.cc:2781] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:47:23.740917 30361 raft_consensus.cc:2454] T a642aecc16f2495a9ca8bac5a93690fa P a4f0bfd4458e4cbd829e905adf720366 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2de2489efa24bc591f6a2963c41d6f8 in term 0.
I20260809 04:47:23.741034 30592 raft_consensus.cc:482] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:47:23.741166 30592 raft_consensus.cc:3037] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:47:23.743459 30592 raft_consensus.cc:504] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2de2489efa24bc591f6a2963c41d6f8" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 36117 } } peers { permanent_uuid: "a4f0bfd4458e4cbd829e905adf720366" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 36001 } } peers { permanent_uuid: "64c3ecf36c9a436b8893781f5130a8cb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 34001 } }
I20260809 04:47:23.744026 30592 leader_election.cc:288] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8 [CANDIDATE]: Term 1 election: Requested vote from peers a4f0bfd4458e4cbd829e905adf720366 (127.12.234.2:36001), 64c3ecf36c9a436b8893781f5130a8cb (127.12.234.1:34001)
I20260809 04:47:23.744402 30361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a642aecc16f2495a9ca8bac5a93690fa" candidate_uuid: "f2de2489efa24bc591f6a2963c41d6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4f0bfd4458e4cbd829e905adf720366"
I20260809 04:47:23.744419 30203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a642aecc16f2495a9ca8bac5a93690fa" candidate_uuid: "f2de2489efa24bc591f6a2963c41d6f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64c3ecf36c9a436b8893781f5130a8cb"
I20260809 04:47:23.744606 30203 raft_consensus.cc:3037] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:47:23.744599 30361 raft_consensus.cc:3037] T a642aecc16f2495a9ca8bac5a93690fa P a4f0bfd4458e4cbd829e905adf720366 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:47:23.746690 30361 raft_consensus.cc:2454] T a642aecc16f2495a9ca8bac5a93690fa P a4f0bfd4458e4cbd829e905adf720366 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2de2489efa24bc591f6a2963c41d6f8 in term 1.
I20260809 04:47:23.746707 30203 raft_consensus.cc:2454] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2de2489efa24bc591f6a2963c41d6f8 in term 1.
I20260809 04:47:23.747121 30441 leader_election.cc:302] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4f0bfd4458e4cbd829e905adf720366, f2de2489efa24bc591f6a2963c41d6f8; no voters:
I20260809 04:47:23.747465 30592 raft_consensus.cc:2781] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:47:23.747924 30592 raft_consensus.cc:686] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8 [term 1 LEADER]: Becoming Leader. State: Replica: f2de2489efa24bc591f6a2963c41d6f8, State: Running, Role: LEADER
I20260809 04:47:23.748381 30592 consensus_queue.cc:237] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2de2489efa24bc591f6a2963c41d6f8" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 36117 } } peers { permanent_uuid: "a4f0bfd4458e4cbd829e905adf720366" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 36001 } } peers { permanent_uuid: "64c3ecf36c9a436b8893781f5130a8cb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 34001 } }
I20260809 04:47:23.752738 30043 catalog_manager.cc:5495] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2de2489efa24bc591f6a2963c41d6f8 (127.12.234.3). New cstate: current_term: 1 leader_uuid: "f2de2489efa24bc591f6a2963c41d6f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2de2489efa24bc591f6a2963c41d6f8" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 36117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4f0bfd4458e4cbd829e905adf720366" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 36001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64c3ecf36c9a436b8893781f5130a8cb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 34001 } health_report { overall_health: UNKNOWN } } }
I20260809 04:47:23.838524 30579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:23.833670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44118 (local address 127.12.234.3:36117)
0809 04:47:23.833847 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:23.833852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:23.833886 (+ 34us) server_negotiation.cc:407] Connection header received
0809 04:47:23.833963 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:23.833966 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:23.834035 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:23.834144 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:47:23.834508 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.835183 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.836747 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.837093 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.837171 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:23.837589 (+ 418us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:47:23.838010 (+ 421us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:47:23.838103 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:23.838199 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 04:47:23.838254 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 04:47:23.845762 30361 raft_consensus.cc:1261] T a642aecc16f2495a9ca8bac5a93690fa P a4f0bfd4458e4cbd829e905adf720366 [term 1 FOLLOWER]: Refusing update from remote peer f2de2489efa24bc591f6a2963c41d6f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:47:23.845897 30203 raft_consensus.cc:1261] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb [term 1 FOLLOWER]: Refusing update from remote peer f2de2489efa24bc591f6a2963c41d6f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:47:23.846789 30595 consensus_queue.cc:1035] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64c3ecf36c9a436b8893781f5130a8cb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 34001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:47:23.847119 30592 consensus_queue.cc:1035] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4f0bfd4458e4cbd829e905adf720366" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 36001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:47:23.875001 30043 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41618:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:47:23.875227 30043 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41618:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:47:23.878496 30043 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98: Sending DeleteTablet for 3 replicas of tablet a642aecc16f2495a9ca8bac5a93690fa
I20260809 04:47:23.879434 30491 tablet_service.cc:1514] Processing DeleteTablet for tablet a642aecc16f2495a9ca8bac5a93690fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:47:23 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:44110
I20260809 04:47:23.879565 30180 tablet_service.cc:1514] Processing DeleteTablet for tablet a642aecc16f2495a9ca8bac5a93690fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:47:23 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:34026
I20260809 04:47:23.879565 30338 tablet_service.cc:1514] Processing DeleteTablet for tablet a642aecc16f2495a9ca8bac5a93690fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:47:23 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:36502
I20260809 04:47:23.880716 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 30112
I20260809 04:47:23.887127 30611 tablet_replica.cc:330] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8: stopping tablet replica
I20260809 04:47:23.887431 30611 raft_consensus.cc:2229] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:47:23.888046 30611 raft_consensus.cc:2258] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:47:23.890065 30611 ts_tablet_manager.cc:1905] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:47:23.897069 30611 ts_tablet_manager.cc:1918] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:47:23.897234 30611 log.cc:1196] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/wal/wals/a642aecc16f2495a9ca8bac5a93690fa
I20260809 04:47:23.897766 30611 ts_tablet_manager.cc:1939] T a642aecc16f2495a9ca8bac5a93690fa P f2de2489efa24bc591f6a2963c41d6f8: Deleting consensus metadata
I20260809 04:47:23.899336 30026 catalog_manager.cc:4841] TS f2de2489efa24bc591f6a2963c41d6f8 (127.12.234.3:36117): tablet a642aecc16f2495a9ca8bac5a93690fa (table test-table [id=825dac4bf1804affa9649e95cfbf09b1]) successfully deleted
I20260809 04:47:23.900806 30612 tablet_replica.cc:330] T a642aecc16f2495a9ca8bac5a93690fa P a4f0bfd4458e4cbd829e905adf720366: stopping tablet replica
I20260809 04:47:23.910679 30612 raft_consensus.cc:2229] T a642aecc16f2495a9ca8bac5a93690fa P a4f0bfd4458e4cbd829e905adf720366 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:47:23.911095 30612 raft_consensus.cc:2258] T a642aecc16f2495a9ca8bac5a93690fa P a4f0bfd4458e4cbd829e905adf720366 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 09 04:47:23 dist-test-slave-1jjb krb5kdc[29992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.1@KRBTEST.COM
I20260809 04:47:23.917635 30612 ts_tablet_manager.cc:1905] T a642aecc16f2495a9ca8bac5a93690fa P a4f0bfd4458e4cbd829e905adf720366: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:47:23.924573 30613 tablet_replica.cc:330] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb: stopping tablet replica
I20260809 04:47:23.924717 30583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:23.881888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34032 (local address 127.12.234.1:34001)
0809 04:47:23.882132 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:23.882142 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:23.900190 (+ 18048us) server_negotiation.cc:407] Connection header received
0809 04:47:23.900402 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:23.900419 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:23.900530 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:23.900646 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:23.903362 (+ 2716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.904875 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.907363 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:23.908288 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:23.908403 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:23.915524 (+ 7121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:23.915543 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:23.915565 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:23.915612 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:23.920291 (+ 4679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:23.921321 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:23.921334 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:23.921339 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:23.921588 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:23.922127 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:23.922135 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:23.922141 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:23.922796 (+ 655us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:23.923152 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:23.923946 (+ 794us) server_negotiation.cc:299] Negotiation successful
0809 04:47:23.924280 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 04:47:23.924949 30613 raft_consensus.cc:2229] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:47:23.925462 30613 raft_consensus.cc:2258] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:47:23.925503 30253 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:47:23.930768 30612 ts_tablet_manager.cc:1918] T a642aecc16f2495a9ca8bac5a93690fa P a4f0bfd4458e4cbd829e905adf720366: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:47:23.930933 30612 log.cc:1196] T a642aecc16f2495a9ca8bac5a93690fa P a4f0bfd4458e4cbd829e905adf720366: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/wal/wals/a642aecc16f2495a9ca8bac5a93690fa
I20260809 04:47:23.931459 30612 ts_tablet_manager.cc:1939] T a642aecc16f2495a9ca8bac5a93690fa P a4f0bfd4458e4cbd829e905adf720366: Deleting consensus metadata
I20260809 04:47:23.933226 30030 catalog_manager.cc:4841] TS a4f0bfd4458e4cbd829e905adf720366 (127.12.234.2:36001): tablet a642aecc16f2495a9ca8bac5a93690fa (table test-table [id=825dac4bf1804affa9649e95cfbf09b1]) successfully deleted
I20260809 04:47:24.012874 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 30112
W20260809 04:47:24.034945 30029 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.234.1:34001 (error 108)
I20260809 04:47:24.035486 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 30265
I20260809 04:47:24.037812 30618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:24.036986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:34001 (local address 127.0.0.1:34038)
0809 04:47:24.037493 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:24.037621 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:34001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":223,"threads_started":1}
W20260809 04:47:24.038082 30029 catalog_manager.cc:4584] TS 64c3ecf36c9a436b8893781f5130a8cb (127.12.234.1:34001): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a642aecc16f2495a9ca8bac5a93690fa: Network error: Client connection negotiation failed: client connection to 127.12.234.1:34001: connect: Connection refused (error 111)
Aug 09 04:47:24 dist-test-slave-1jjb krb5kdc[29992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.2@KRBTEST.COM
I20260809 04:47:24.052790 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:24.036331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36506 (local address 127.12.234.2:36001)
0809 04:47:24.036550 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:24.036556 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:24.036670 (+ 114us) server_negotiation.cc:407] Connection header received
0809 04:47:24.036835 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:24.036840 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:24.036929 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:24.037035 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:24.038700 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:24.039680 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:24.041113 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:24.041719 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:24.041833 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:24.046298 (+ 4465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:24.046314 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:24.046332 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:24.046379 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:24.050034 (+ 3655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:24.050709 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:24.050717 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:24.050721 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:24.050861 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:24.051208 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:24.051212 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:24.051216 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:24.051641 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:24.051893 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:24.052211 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 04:47:24.052431 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 04:47:24.053519 30407 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:47:24.091282 30618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:24.090915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:34001 (local address 127.0.0.1:34048)
0809 04:47:24.091094 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:24.091168 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:34001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 04:47:24.092346 30618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:24.091953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:34001 (local address 127.0.0.1:34058)
0809 04:47:24.092153 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:24.092218 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:34001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 04:47:24.153054 30618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:24.152522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:34001 (local address 127.0.0.1:34074)
0809 04:47:24.152748 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:24.152876 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:34001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260809 04:47:24.154255 30618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:24.153836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:34001 (local address 127.0.0.1:34086)
0809 04:47:24.154006 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:24.154111 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:34001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 04:47:24.171784 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 30265
I20260809 04:47:24.192761 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 30419
Aug 09 04:47:24 dist-test-slave-1jjb krb5kdc[29992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250841, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.3@KRBTEST.COM
I20260809 04:47:24.210296 30579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:24.193560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44134 (local address 127.12.234.3:36117)
0809 04:47:24.193777 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:24.193783 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:24.193805 (+ 22us) server_negotiation.cc:407] Connection header received
0809 04:47:24.193920 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:24.193925 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:24.194014 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:24.194146 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:24.195468 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:24.196724 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:24.197869 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:24.198535 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:24.198677 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:24.203203 (+ 4526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:24.203235 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:24.203258 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:24.203305 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:24.207266 (+ 3961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:24.208030 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:24.208041 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:24.208044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:24.208265 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:24.208656 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:24.208662 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:24.208665 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:24.209171 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:24.209494 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:24.209654 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 04:47:24.209882 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 04:47:24.211163 30558 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:47:24.234724 30618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:24.234326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:34001 (local address 127.0.0.1:34094)
0809 04:47:24.234501 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:24.234594 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:34001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 04:47:24.235805 30618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:24.235407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:34001 (local address 127.0.0.1:34110)
0809 04:47:24.235602 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:24.235679 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:34001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260809 04:47:24.328881 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 30419
I20260809 04:47:24.352734 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 30008
I20260809 04:47:24.354086 30076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:47:24.459296 30618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:24.458586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:34001 (local address 127.0.0.1:34116)
0809 04:47:24.458875 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:24.459030 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:34001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 04:47:24.459764 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 30008
I20260809 04:47:44.487396 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:40967
--webserver_interface=127.12.234.62
--webserver_port=38989
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:42447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:40967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:47:44.810037 30623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:44.810308 30623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:44.810388 30623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:44.817016 30623 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 04:47:44.817095 30623 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 04:47:44.817168 30623 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 04:47:44.817265 30623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:47:44.817358 30623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:44.817482 30623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:44.817559 30623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:47:44.817647 30623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:47:44.830542 30623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:42447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:40967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:40967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=38989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:47:44.832352 30623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:44.833621 30623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:44.863449 30629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:44.863420 30630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:44.864367 30632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:44.864696 30623 server_base.cc:1034] running on GCE node
I20260809 04:47:44.865592 30623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:44.867238 30623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:44.868522 30623 hybrid_clock.cc:648] HybridClock initialized: now 1786250864868443 us; error 42 us; skew 500 ppm
I20260809 04:47:44.868939 30623 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:47:44 dist-test-slave-1jjb krb5kdc[29992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250864, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:44.872587 30623 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:47:44.874291 30623 webserver.cc:460] Webserver started at http://127.12.234.62:38989/ using document root <none> and password file <none>
I20260809 04:47:44.875067 30623 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:44.875205 30623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:44.880765 30623 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:47:44.883401 30640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:44.884692 30623 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:47:44.884884 30623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "5695b9eea47c429d98e81934ebd75f98"
format_stamp: "Formatted at 2026-08-09 04:47:21 on dist-test-slave-1jjb"
server_key: "195c367a48003c7baa84a860dca6a8e5"
server_key_iv: "e11c3bafb2da7448cbdbde76601618b0"
server_key_version: "encryptionkey@0"
I20260809 04:47:44.885850 30623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:44.902038 30623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:44.905826 30623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:44.906107 30623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:44.924315 30623 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:40967
I20260809 04:47:44.924345 30702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:40967 every 8 connection(s)
I20260809 04:47:44.925663 30623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:47:44.927704 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 30623
I20260809 04:47:44.928776 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:34001
--local_ip_for_outbound_sockets=127.12.234.1
--tserver_master_addrs=127.12.234.62:40967
--webserver_port=40791
--webserver_interface=127.12.234.1
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:42447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:47:44.934998 30703 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:47:44.939774 30703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98: Bootstrap starting.
I20260809 04:47:44.959017 30703 log.cc:824] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:44.970752 30703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:47:44.971378 30703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98: Bootstrap complete.
I20260809 04:47:44.980613 30703 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695b9eea47c429d98e81934ebd75f98" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 40967 } }
I20260809 04:47:44.981451 30703 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5695b9eea47c429d98e81934ebd75f98, State: Initialized, Role: FOLLOWER
I20260809 04:47:44.982161 30703 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695b9eea47c429d98e81934ebd75f98" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 40967 } }
I20260809 04:47:44.982414 30703 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:47:44.982527 30703 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:47:44.982717 30703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [term 1 FOLLOWER]: Advancing to term 2
I20260809 04:47:44.986479 30703 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695b9eea47c429d98e81934ebd75f98" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 40967 } }
I20260809 04:47:44.986991 30703 leader_election.cc:302] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5695b9eea47c429d98e81934ebd75f98; no voters:
I20260809 04:47:44.987612 30703 leader_election.cc:288] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 04:47:44.987737 30707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 04:47:44.988969 30707 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [term 2 LEADER]: Becoming Leader. State: Replica: 5695b9eea47c429d98e81934ebd75f98, State: Running, Role: LEADER
I20260809 04:47:44.989667 30707 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695b9eea47c429d98e81934ebd75f98" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 40967 } }
I20260809 04:47:44.990056 30703 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:47:44.992379 30708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5695b9eea47c429d98e81934ebd75f98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695b9eea47c429d98e81934ebd75f98" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 40967 } } }
I20260809 04:47:44.992810 30708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [sys.catalog]: This master's current role is: LEADER
I20260809 04:47:44.994333 30713 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:47:44.994753 30709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5695b9eea47c429d98e81934ebd75f98. Latest consensus state: current_term: 2 leader_uuid: "5695b9eea47c429d98e81934ebd75f98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695b9eea47c429d98e81934ebd75f98" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 40967 } } }
I20260809 04:47:44.995175 30709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [sys.catalog]: This master's current role is: LEADER
I20260809 04:47:45.002347 30713 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:47:45.003667 30713 catalog_manager.cc:1259] Loaded cluster ID: 011cef416d964908a136759b2817a2ae
I20260809 04:47:45.003826 30713 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:47:45.007774 30713 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:47:45.009239 30713 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98: Loaded TSK: 0
I20260809 04:47:45.041066 30713 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98: Generated new TSK 1
I20260809 04:47:45.041395 30713 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 04:47:45.266422 30705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:45.266690 30705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:45.266841 30705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:45.273862 30705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:45.274025 30705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:45.274190 30705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:47:45.283452 30705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:42447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:34001
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=40791
--tserver_master_addrs=127.12.234.62:40967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:47:45.284720 30705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:45.286080 30705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:45.300915 30734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:45.302474 30736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:45.302614 30733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:45.303596 30705 server_base.cc:1034] running on GCE node
I20260809 04:47:45.304430 30705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:45.305660 30705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:45.306910 30705 hybrid_clock.cc:648] HybridClock initialized: now 1786250865306860 us; error 54 us; skew 500 ppm
I20260809 04:47:45.307273 30705 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:47:45 dist-test-slave-1jjb krb5kdc[29992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250865, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:45.310901 30705 init.cc:377] Logged in from keytab as kudu/127.12.234.1@KRBTEST.COM (short username kudu)
I20260809 04:47:45.312333 30705 webserver.cc:460] Webserver started at http://127.12.234.1:40791/ using document root <none> and password file <none>
I20260809 04:47:45.312950 30705 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:45.313094 30705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:45.318339 30705 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 04:47:45.321187 30745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:45.322487 30705 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:47:45.322649 30705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "64c3ecf36c9a436b8893781f5130a8cb"
format_stamp: "Formatted at 2026-08-09 04:47:22 on dist-test-slave-1jjb"
server_key: "553137d64b696ca9eb85cd1d45699e12"
server_key_iv: "3937e74f3c5c6686ec9611523cbe2752"
server_key_version: "encryptionkey@0"
I20260809 04:47:45.323617 30705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:45.346903 30705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:45.351555 30705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:45.351821 30705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:45.352953 30705 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:47:45.354969 30753 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260809 04:47:45.360276 30705 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 04:47:45.360374 30705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260809 04:47:45.360462 30705 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 04:47:45.363214 30705 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 04:47:45.363279 30705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20260809 04:47:45.363814 30753 tablet_bootstrap.cc:492] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb: Bootstrap starting.
I20260809 04:47:45.372056 30753 log.cc:824] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:45.385911 30753 tablet_bootstrap.cc:492] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:47:45.386759 30753 tablet_bootstrap.cc:492] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb: Bootstrap complete.
I20260809 04:47:45.387269 30753 ts_tablet_manager.cc:1397] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb: Time spent bootstrapping tablet: real 0.024s user 0.015s sys 0.007s
I20260809 04:47:45.398043 30753 raft_consensus.cc:348] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2de2489efa24bc591f6a2963c41d6f8" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 36117 } } peers { permanent_uuid: "a4f0bfd4458e4cbd829e905adf720366" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 36001 } } peers { permanent_uuid: "64c3ecf36c9a436b8893781f5130a8cb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 34001 } }
I20260809 04:47:45.399545 30753 raft_consensus.cc:729] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64c3ecf36c9a436b8893781f5130a8cb, State: Initialized, Role: FOLLOWER
I20260809 04:47:45.400439 30753 consensus_queue.cc:260] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2de2489efa24bc591f6a2963c41d6f8" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 36117 } } peers { permanent_uuid: "a4f0bfd4458e4cbd829e905adf720366" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 36001 } } peers { permanent_uuid: "64c3ecf36c9a436b8893781f5130a8cb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 34001 } }
I20260809 04:47:45.402006 30753 ts_tablet_manager.cc:1428] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260809 04:47:45.402314 30705 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:34001
I20260809 04:47:45.402390 30878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:34001 every 8 connection(s)
I20260809 04:47:45.403438 30705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:47:45.409938 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 30705
I20260809 04:47:45.410590 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:36001
--local_ip_for_outbound_sockets=127.12.234.2
--tserver_master_addrs=127.12.234.62:40967
--webserver_port=34997
--webserver_interface=127.12.234.2
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:42447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:47:45 dist-test-slave-1jjb krb5kdc[29992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250865, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:47:45.423563 30882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:45.406073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.1:56733)
0809 04:47:45.406900 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:45.406956 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:45.407362 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:45.408578 (+ 1216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:45.408584 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:45.409200 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:45.409782 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:45.409798 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:45.411480 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:45.411484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:45.412733 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:45.412744 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:45.413633 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:45.413639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:45.413924 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:45.414614 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:45.414635 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:45.416619 (+ 1984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:45.419716 (+ 3097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:45.419724 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:45.419736 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:45.420201 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:45.420628 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:45.420633 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:45.420636 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:45.420795 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:45.421472 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:45.421479 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:45.421758 (+ 279us) client_negotiation.cc:769] Sending connection context
0809 04:47:45.422004 (+ 246us) client_negotiation.cc:240] Negotiation successful
0809 04:47:45.422447 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":366,"threads_started":1}
I20260809 04:47:45.423559 30881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:45.406134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:56733 (local address 127.12.234.62:40967)
0809 04:47:45.406671 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:45.406681 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:45.407069 (+ 388us) server_negotiation.cc:407] Connection header received
0809 04:47:45.407794 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:45.407805 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:45.408187 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:45.408450 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:45.409989 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:45.411315 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:45.412870 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:45.413436 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:45.413639 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:45.416883 (+ 3244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:45.416891 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:45.416903 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:45.416947 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:45.419581 (+ 2634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:45.420320 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:45.420332 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:45.420337 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:45.420507 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:45.420935 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:45.420944 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:45.420946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:45.421340 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:45.421514 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:45.422231 (+ 717us) server_negotiation.cc:299] Negotiation successful
0809 04:47:45.422574 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":242,"threads_started":1}
I20260809 04:47:45.426563 30879 heartbeater.cc:347] Connected to a master server at 127.12.234.62:40967
I20260809 04:47:45.426867 30879 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:45.427513 30879 heartbeater.cc:511] Master 127.12.234.62:40967 requested a full tablet report, sending...
I20260809 04:47:45.430330 30663 ts_manager.cc:194] Registered new tserver with Master: 64c3ecf36c9a436b8893781f5130a8cb (127.12.234.1:34001)
I20260809 04:47:45.434616 30663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.1@KRBTEST.COM'} at 127.12.234.1:56733
Aug 09 04:47:45 dist-test-slave-1jjb krb5kdc[29992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250864, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for kudu/127.12.234.1@KRBTEST.COM
I20260809 04:47:45.449717 30885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:45.433314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:34001 (local address 127.0.0.1:32846)
0809 04:47:45.436993 (+ 3679us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:45.437030 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:45.437203 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:45.437751 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:45.437757 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:45.438409 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:45.438661 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:45.438670 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:45.439938 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:45.439941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:45.440845 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:45.440857 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:45.441631 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:45.441637 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:45.441808 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:45.442375 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:45.442391 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:45.444393 (+ 2002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:45.447239 (+ 2846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:45.447250 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:45.447255 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:45.447595 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:45.447946 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:45.447950 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:45.447953 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:45.448118 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:45.448667 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:45.448673 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:45.448835 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 04:47:45.448994 (+ 159us) client_negotiation.cc:240] Negotiation successful
0809 04:47:45.449288 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3417,"thread_start_us":3266,"threads_started":1}
I20260809 04:47:45.449800 30887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:45.433449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32846 (local address 127.12.234.1:34001)
0809 04:47:45.436619 (+ 3170us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:45.436626 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:45.437178 (+ 552us) server_negotiation.cc:407] Connection header received
0809 04:47:45.437397 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:45.437405 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:45.437588 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:45.437723 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:45.438831 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:45.439777 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:45.440983 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:45.441462 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:45.441606 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:45.444617 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:45.444628 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:45.444638 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:45.444664 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:45.447103 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:45.447730 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:45.447736 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:45.447737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:45.447838 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:45.448231 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:45.448241 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:45.448245 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:45.448541 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:45.448715 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:45.449150 (+ 435us) server_negotiation.cc:299] Negotiation successful
0809 04:47:45.449433 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2989,"thread_start_us":2808,"threads_started":1}
I20260809 04:47:45.451038 30804 tablet_service.cc:1514] Processing DeleteTablet for tablet a642aecc16f2495a9ca8bac5a93690fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:47:23 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:32846
I20260809 04:47:45.452973 30888 tablet_replica.cc:330] stopping tablet replica
I20260809 04:47:45.453377 30888 raft_consensus.cc:2229] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:47:45.453794 30888 raft_consensus.cc:2258] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:47:45.455196 30888 ts_tablet_manager.cc:1905] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:47:45.464619 30888 ts_tablet_manager.cc:1918] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:47:45.464800 30888 log.cc:1196] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/wal/wals/a642aecc16f2495a9ca8bac5a93690fa
I20260809 04:47:45.465338 30888 ts_tablet_manager.cc:1939] T a642aecc16f2495a9ca8bac5a93690fa P 64c3ecf36c9a436b8893781f5130a8cb: Deleting consensus metadata
I20260809 04:47:45.467500 30643 catalog_manager.cc:4841] TS 64c3ecf36c9a436b8893781f5130a8cb (127.12.234.1:34001): tablet a642aecc16f2495a9ca8bac5a93690fa (table test-table [id=825dac4bf1804affa9649e95cfbf09b1]) successfully deleted
W20260809 04:47:45.733532 30884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:45.733762 30884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:45.733912 30884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:45.740095 30884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:45.740175 30884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:45.740316 30884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:47:45.748659 30884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:42447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:36001
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=34997
--tserver_master_addrs=127.12.234.62:40967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:47:45.749749 30884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:45.750914 30884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:45.763113 30898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:45.763198 30895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:45.763177 30894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:45.764315 30884 server_base.cc:1034] running on GCE node
I20260809 04:47:45.764849 30884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:45.765614 30884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:45.766793 30884 hybrid_clock.cc:648] HybridClock initialized: now 1786250865766741 us; error 45 us; skew 500 ppm
I20260809 04:47:45.767047 30884 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:47:45 dist-test-slave-1jjb krb5kdc[29992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250865, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:45.769804 30884 init.cc:377] Logged in from keytab as kudu/127.12.234.2@KRBTEST.COM (short username kudu)
I20260809 04:47:45.771059 30884 webserver.cc:460] Webserver started at http://127.12.234.2:34997/ using document root <none> and password file <none>
I20260809 04:47:45.771685 30884 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:45.771824 30884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:45.776669 30884 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:47:45.779124 30906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:45.780251 30884 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:47:45.780400 30884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "a4f0bfd4458e4cbd829e905adf720366"
format_stamp: "Formatted at 2026-08-09 04:47:22 on dist-test-slave-1jjb"
server_key: "9261fad4449bd10e24901bcb0a3b1955"
server_key_iv: "10ca47b25d37ebc037d9d5468c7dc75e"
server_key_version: "encryptionkey@0"
I20260809 04:47:45.781359 30884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:45.813585 30884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:45.817559 30884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:45.817809 30884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:45.818701 30884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:47:45.819904 30884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:47:45.820004 30884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:45.820143 30884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:47:45.820251 30884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:45.851696 30884 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:36001
I20260809 04:47:45.851791 31032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:36001 every 8 connection(s)
I20260809 04:47:45.853096 30884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:47:45.858011 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 30884
I20260809 04:47:45.858716 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:36117
--local_ip_for_outbound_sockets=127.12.234.3
--tserver_master_addrs=127.12.234.62:40967
--webserver_port=33145
--webserver_interface=127.12.234.3
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:42447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 04:47:45 dist-test-slave-1jjb krb5kdc[29992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250865, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:47:45.873059 30881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:45.855549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:52833 (local address 127.12.234.62:40967)
0809 04:47:45.855740 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:45.855746 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:45.856913 (+ 1167us) server_negotiation.cc:407] Connection header received
0809 04:47:45.858704 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:45.858712 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:45.858800 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:45.858914 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:45.860403 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:45.861477 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:45.862969 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:45.863676 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:45.863831 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:45.867557 (+ 3726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:45.867561 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:45.867564 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:45.867594 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:45.869988 (+ 2394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:45.870698 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:45.870706 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:45.870710 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:45.870856 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:45.871232 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:45.871240 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:45.871244 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:45.871601 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:45.871746 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:45.872370 (+ 624us) server_negotiation.cc:299] Negotiation successful
0809 04:47:45.872617 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 04:47:45.873548 31036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:45.856020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.2:52833)
0809 04:47:45.856757 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:45.856797 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:45.858437 (+ 1640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:45.859198 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:45.859205 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:45.859660 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:45.860239 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:45.860258 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:45.861642 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:45.861645 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:45.862804 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:45.862816 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:45.863870 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:45.863875 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:45.864115 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:45.864794 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:45.864814 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:45.867339 (+ 2525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:45.870142 (+ 2803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:45.870152 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:45.870168 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:45.870574 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:45.870960 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:45.870962 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:45.870964 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:45.871120 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:45.871717 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:45.871721 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:45.871985 (+ 264us) client_negotiation.cc:769] Sending connection context
0809 04:47:45.872240 (+ 255us) client_negotiation.cc:240] Negotiation successful
0809 04:47:45.872553 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":266,"threads_started":1}
I20260809 04:47:45.874926 31033 heartbeater.cc:347] Connected to a master server at 127.12.234.62:40967
I20260809 04:47:45.875233 31033 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:45.875938 31033 heartbeater.cc:511] Master 127.12.234.62:40967 requested a full tablet report, sending...
I20260809 04:47:45.877342 30663 ts_manager.cc:194] Registered new tserver with Master: a4f0bfd4458e4cbd829e905adf720366 (127.12.234.2:36001)
I20260809 04:47:45.878099 30663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.2@KRBTEST.COM'} at 127.12.234.2:52833
W20260809 04:47:46.168376 31037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:47:46.168634 31037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:47:46.168789 31037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:47:46.175473 31037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:47:46.175614 31037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:47:46.175776 31037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:47:46.184312 31037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:42447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:36117
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=33145
--tserver_master_addrs=127.12.234.62:40967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:47:46.185557 31037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:47:46.186643 31037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:47:46.199339 31046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:46.199301 31044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:47:46.199380 31043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:47:46.201345 31037 server_base.cc:1034] running on GCE node
I20260809 04:47:46.201908 31037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:47:46.202661 31037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:47:46.203879 31037 hybrid_clock.cc:648] HybridClock initialized: now 1786250866203842 us; error 55 us; skew 500 ppm
I20260809 04:47:46.204268 31037 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:47:46 dist-test-slave-1jjb krb5kdc[29992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250866, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:47:46.207345 31037 init.cc:377] Logged in from keytab as kudu/127.12.234.3@KRBTEST.COM (short username kudu)
I20260809 04:47:46.208591 31037 webserver.cc:460] Webserver started at http://127.12.234.3:33145/ using document root <none> and password file <none>
I20260809 04:47:46.209205 31037 fs_manager.cc:359] Metadata directory not provided
I20260809 04:47:46.209306 31037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:47:46.213899 31037 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 04:47:46.216356 31053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:46.217567 31037 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:47:46.217697 31037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "f2de2489efa24bc591f6a2963c41d6f8"
format_stamp: "Formatted at 2026-08-09 04:47:23 on dist-test-slave-1jjb"
server_key: "fc7dca100bf153b7bad0f2f4a5ab64da"
server_key_iv: "2edc1153d99e56082098e456cbe1c05a"
server_key_version: "encryptionkey@0"
I20260809 04:47:46.218557 31037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:47:46.232292 31037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:47:46.236068 31037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:47:46.236346 31037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:47:46.237165 31037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:47:46.238217 31037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:47:46.238292 31037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:46.238360 31037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:47:46.238411 31037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:47:46.268867 31037 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:36117
I20260809 04:47:46.268944 31181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:36117 every 8 connection(s)
I20260809 04:47:46.270187 31037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:47:46.277985 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 31037
Aug 09 04:47:46 dist-test-slave-1jjb krb5kdc[29992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250866, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:47:46.289381 30881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:46.272523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:51783 (local address 127.12.234.62:40967)
0809 04:47:46.272694 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:46.272698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:46.273699 (+ 1001us) server_negotiation.cc:407] Connection header received
0809 04:47:46.275527 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:46.275533 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:46.275617 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:46.275721 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:46.277219 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.278055 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.279922 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.280383 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.280497 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:46.283432 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:46.283439 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:46.283445 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:46.283480 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:46.286320 (+ 2840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:46.286991 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:46.286998 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:46.287001 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:46.287129 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:46.287673 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:46.287679 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:46.287682 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:46.287960 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:46.288156 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:46.288771 (+ 615us) server_negotiation.cc:299] Negotiation successful
0809 04:47:46.289020 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:47:46.290419 31185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:46.272893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.3:51783)
0809 04:47:46.273547 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:46.273581 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:46.275228 (+ 1647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:46.275985 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:46.275995 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:46.276509 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:47:46.277021 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:46.277038 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.278209 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.278213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:46.279787 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:46.279799 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.280591 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.280598 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:46.280776 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:46.281350 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:47:46.281368 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:47:46.283233 (+ 1865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:47:46.286496 (+ 3263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:46.286504 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:46.286516 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:46.286877 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:46.287352 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:47:46.287358 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:47:46.287362 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:47:46.287554 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:47:46.288117 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:47:46.288124 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:47:46.288453 (+ 329us) client_negotiation.cc:769] Sending connection context
0809 04:47:46.288692 (+ 239us) client_negotiation.cc:240] Negotiation successful
0809 04:47:46.289250 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":261,"threads_started":1}
I20260809 04:47:46.291878 31182 heartbeater.cc:347] Connected to a master server at 127.12.234.62:40967
I20260809 04:47:46.292239 31182 heartbeater.cc:464] Registering TS with master...
I20260809 04:47:46.293098 31186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:46.278855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51326 (local address 127.12.234.62:40967)
0809 04:47:46.279242 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:46.279245 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:46.279669 (+ 424us) server_negotiation.cc:407] Connection header received
0809 04:47:46.279840 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:46.279844 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:46.279913 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:46.280045 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:46.281570 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.282222 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.283463 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.284201 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.284354 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:46.287134 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:46.287138 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:46.287141 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:46.287168 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:46.290278 (+ 3110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:46.291012 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:46.291021 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:46.291023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:46.291131 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:46.291486 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:46.291491 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:46.291494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:46.291827 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:46.292110 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:46.292525 (+ 415us) server_negotiation.cc:299] Negotiation successful
0809 04:47:46.292738 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":238,"threads_started":1}
I20260809 04:47:46.293115 31182 heartbeater.cc:511] Master 127.12.234.62:40967 requested a full tablet report, sending...
I20260809 04:47:46.294621 30663 ts_manager.cc:194] Registered new tserver with Master: f2de2489efa24bc591f6a2963c41d6f8 (127.12.234.3:36117)
I20260809 04:47:46.295531 30663 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.3@KRBTEST.COM'} at 127.12.234.3:51783
I20260809 04:47:46.295794 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:47:46.302122 31186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:46.296878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51342 (local address 127.12.234.62:40967)
0809 04:47:46.297052 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:46.297058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:46.297593 (+ 535us) server_negotiation.cc:407] Connection header received
0809 04:47:46.297802 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:46.297807 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:46.297945 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:46.298048 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:47:46.298483 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.299087 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.300532 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.300887 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.300980 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:46.301350 (+ 370us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:47:46.301563 (+ 213us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 04:47:46.301883 (+ 320us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51342: token expired
Metrics: {"server-negotiator.queue_time_us":34}
W20260809 04:47:46.302253 31186 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51342: token expired
W20260809 04:47:46.302824 30570 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.12.234.62:40967)
I20260809 04:47:46.314611 31186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:46.303418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51356 (local address 127.12.234.62:40967)
0809 04:47:46.303587 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:46.303592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:46.303610 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:47:46.303699 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:46.303703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:46.303760 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:46.303826 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:46.305479 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.306076 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.307290 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.307624 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.307706 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:46.310187 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:46.310191 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:46.310194 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:46.310233 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:46.312458 (+ 2225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:46.313025 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:46.313029 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:46.313031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:46.313165 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:46.313507 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:46.313512 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:46.313516 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:46.313757 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:46.313933 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:46.314191 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 04:47:46.314342 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 04:47:46.317807 30663 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:47:46.325090 30663 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:47:46.337646 30804 tablet_service.cc:1467] Processing CreateTablet for tablet 8ee831722a22446db27a55e905d9f05f (DEFAULT_TABLE table=test-table [id=6f9db36101a34389acbf0954ebc35e46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:47:46.338464 30804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ee831722a22446db27a55e905d9f05f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:47:46.343355 31195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:46.334741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:36001 (local address 127.0.0.1:38934)
0809 04:47:46.335357 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:46.335378 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:46.335493 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:46.336799 (+ 1306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:46.336803 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:46.336834 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:46.337126 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:46.337139 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.338694 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.338699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:46.340474 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:46.340487 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.342310 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.342316 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:46.342677 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:46.342738 (+ 61us) client_negotiation.cc:769] Sending connection context
0809 04:47:46.342830 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 04:47:46.342957 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":333,"threads_started":1}
I20260809 04:47:46.344156 31197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:46.335248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38934 (local address 127.12.234.2:36001)
0809 04:47:46.335829 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:46.335839 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:46.335877 (+ 38us) server_negotiation.cc:407] Connection header received
0809 04:47:46.335992 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:46.336001 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:46.336285 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:46.336496 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:46.337332 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.338544 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.340640 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.342109 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.342245 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:46.342501 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:46.343576 (+ 1075us) server_negotiation.cc:299] Negotiation successful
0809 04:47:46.343757 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":269,"threads_started":1}
I20260809 04:47:46.344470 31198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:46.335240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:36117 (local address 127.0.0.1:40644)
0809 04:47:46.336135 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:46.336153 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:46.336263 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:46.338245 (+ 1982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:46.338250 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:46.338273 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:46.338574 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:46.338588 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.340369 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.340375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:46.341718 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:46.341731 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.343684 (+ 1953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.343691 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:46.343864 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:46.343898 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 04:47:46.344003 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 04:47:46.344157 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":764,"thread_start_us":890,"threads_started":1}
I20260809 04:47:46.344878 31199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:46.335835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40644 (local address 127.12.234.3:36117)
0809 04:47:46.337736 (+ 1901us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:46.337743 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:46.337773 (+ 30us) server_negotiation.cc:407] Connection header received
0809 04:47:46.337863 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:46.337870 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:46.338050 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:46.338240 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:46.338752 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.340211 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.341884 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.343167 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.343305 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:46.343543 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:46.344194 (+ 651us) server_negotiation.cc:299] Negotiation successful
0809 04:47:46.344550 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1744,"thread_start_us":1618,"threads_started":1}
I20260809 04:47:46.346709 31200 tablet_bootstrap.cc:492] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb: Bootstrap starting.
I20260809 04:47:46.348099 31107 tablet_service.cc:1467] Processing CreateTablet for tablet 8ee831722a22446db27a55e905d9f05f (DEFAULT_TABLE table=test-table [id=6f9db36101a34389acbf0954ebc35e46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:47:46.347970 30960 tablet_service.cc:1467] Processing CreateTablet for tablet 8ee831722a22446db27a55e905d9f05f (DEFAULT_TABLE table=test-table [id=6f9db36101a34389acbf0954ebc35e46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:47:46.348855 31200 tablet_bootstrap.cc:654] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb: Neither blocks nor log segments found. Creating new log.
I20260809 04:47:46.349500 31107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ee831722a22446db27a55e905d9f05f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:47:46.349500 30960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ee831722a22446db27a55e905d9f05f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:47:46.350673 31200 tablet_bootstrap.cc:492] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb: No bootstrap required, opened a new log
I20260809 04:47:46.350834 31200 ts_tablet_manager.cc:1397] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 04:47:46.351452 31200 raft_consensus.cc:348] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f0bfd4458e4cbd829e905adf720366" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 36001 } } peers { permanent_uuid: "f2de2489efa24bc591f6a2963c41d6f8" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 36117 } } peers { permanent_uuid: "64c3ecf36c9a436b8893781f5130a8cb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 34001 } }
I20260809 04:47:46.351675 31200 raft_consensus.cc:374] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:47:46.351797 31200 raft_consensus.cc:729] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64c3ecf36c9a436b8893781f5130a8cb, State: Initialized, Role: FOLLOWER
I20260809 04:47:46.352072 31200 consensus_queue.cc:260] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f0bfd4458e4cbd829e905adf720366" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 36001 } } peers { permanent_uuid: "f2de2489efa24bc591f6a2963c41d6f8" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 36117 } } peers { permanent_uuid: "64c3ecf36c9a436b8893781f5130a8cb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 34001 } }
I20260809 04:47:46.356621 30879 heartbeater.cc:503] Master 127.12.234.62:40967 was elected leader, sending a full tablet report...
I20260809 04:47:46.356621 31200 ts_tablet_manager.cc:1428] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb: Time spent starting tablet: real 0.006s user 0.000s sys 0.006s
I20260809 04:47:46.364141 31203 tablet_bootstrap.cc:492] T 8ee831722a22446db27a55e905d9f05f P a4f0bfd4458e4cbd829e905adf720366: Bootstrap starting.
I20260809 04:47:46.364836 31205 tablet_bootstrap.cc:492] T 8ee831722a22446db27a55e905d9f05f P f2de2489efa24bc591f6a2963c41d6f8: Bootstrap starting.
I20260809 04:47:46.367944 31203 tablet_bootstrap.cc:654] T 8ee831722a22446db27a55e905d9f05f P a4f0bfd4458e4cbd829e905adf720366: Neither blocks nor log segments found. Creating new log.
I20260809 04:47:46.368281 31205 tablet_bootstrap.cc:654] T 8ee831722a22446db27a55e905d9f05f P f2de2489efa24bc591f6a2963c41d6f8: Neither blocks nor log segments found. Creating new log.
I20260809 04:47:46.369237 31203 log.cc:824] T 8ee831722a22446db27a55e905d9f05f P a4f0bfd4458e4cbd829e905adf720366: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:46.369387 31205 log.cc:824] T 8ee831722a22446db27a55e905d9f05f P f2de2489efa24bc591f6a2963c41d6f8: Log is configured to *not* fsync() on all Append() calls
I20260809 04:47:46.372303 31203 tablet_bootstrap.cc:492] T 8ee831722a22446db27a55e905d9f05f P a4f0bfd4458e4cbd829e905adf720366: No bootstrap required, opened a new log
I20260809 04:47:46.372304 31205 tablet_bootstrap.cc:492] T 8ee831722a22446db27a55e905d9f05f P f2de2489efa24bc591f6a2963c41d6f8: No bootstrap required, opened a new log
I20260809 04:47:46.372543 31205 ts_tablet_manager.cc:1397] T 8ee831722a22446db27a55e905d9f05f P f2de2489efa24bc591f6a2963c41d6f8: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 04:47:46.372543 31203 ts_tablet_manager.cc:1397] T 8ee831722a22446db27a55e905d9f05f P a4f0bfd4458e4cbd829e905adf720366: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260809 04:47:46.382464 31205 raft_consensus.cc:348] T 8ee831722a22446db27a55e905d9f05f P f2de2489efa24bc591f6a2963c41d6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f0bfd4458e4cbd829e905adf720366" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 36001 } } peers { permanent_uuid: "f2de2489efa24bc591f6a2963c41d6f8" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 36117 } } peers { permanent_uuid: "64c3ecf36c9a436b8893781f5130a8cb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 34001 } }
I20260809 04:47:46.382454 31203 raft_consensus.cc:348] T 8ee831722a22446db27a55e905d9f05f P a4f0bfd4458e4cbd829e905adf720366 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f0bfd4458e4cbd829e905adf720366" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 36001 } } peers { permanent_uuid: "f2de2489efa24bc591f6a2963c41d6f8" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 36117 } } peers { permanent_uuid: "64c3ecf36c9a436b8893781f5130a8cb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 34001 } }
I20260809 04:47:46.382848 31203 raft_consensus.cc:374] T 8ee831722a22446db27a55e905d9f05f P a4f0bfd4458e4cbd829e905adf720366 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:47:46.382848 31205 raft_consensus.cc:374] T 8ee831722a22446db27a55e905d9f05f P f2de2489efa24bc591f6a2963c41d6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:47:46.382958 31203 raft_consensus.cc:729] T 8ee831722a22446db27a55e905d9f05f P a4f0bfd4458e4cbd829e905adf720366 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4f0bfd4458e4cbd829e905adf720366, State: Initialized, Role: FOLLOWER
I20260809 04:47:46.382949 31205 raft_consensus.cc:729] T 8ee831722a22446db27a55e905d9f05f P f2de2489efa24bc591f6a2963c41d6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2de2489efa24bc591f6a2963c41d6f8, State: Initialized, Role: FOLLOWER
I20260809 04:47:46.383556 31205 consensus_queue.cc:260] T 8ee831722a22446db27a55e905d9f05f P f2de2489efa24bc591f6a2963c41d6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f0bfd4458e4cbd829e905adf720366" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 36001 } } peers { permanent_uuid: "f2de2489efa24bc591f6a2963c41d6f8" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 36117 } } peers { permanent_uuid: "64c3ecf36c9a436b8893781f5130a8cb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 34001 } }
I20260809 04:47:46.383555 31203 consensus_queue.cc:260] T 8ee831722a22446db27a55e905d9f05f P a4f0bfd4458e4cbd829e905adf720366 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f0bfd4458e4cbd829e905adf720366" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 36001 } } peers { permanent_uuid: "f2de2489efa24bc591f6a2963c41d6f8" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 36117 } } peers { permanent_uuid: "64c3ecf36c9a436b8893781f5130a8cb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 34001 } }
I20260809 04:47:46.384706 31033 heartbeater.cc:503] Master 127.12.234.62:40967 was elected leader, sending a full tablet report...
I20260809 04:47:46.385203 31182 heartbeater.cc:503] Master 127.12.234.62:40967 was elected leader, sending a full tablet report...
I20260809 04:47:46.385780 31205 ts_tablet_manager.cc:1428] T 8ee831722a22446db27a55e905d9f05f P f2de2489efa24bc591f6a2963c41d6f8: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260809 04:47:46.385983 31203 ts_tablet_manager.cc:1428] T 8ee831722a22446db27a55e905d9f05f P a4f0bfd4458e4cbd829e905adf720366: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
W20260809 04:47:46.406289 30880 tablet.cc:2369] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:47:46.451560 31202 raft_consensus.cc:482] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:47:46.451768 31202 raft_consensus.cc:504] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f0bfd4458e4cbd829e905adf720366" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 36001 } } peers { permanent_uuid: "f2de2489efa24bc591f6a2963c41d6f8" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 36117 } } peers { permanent_uuid: "64c3ecf36c9a436b8893781f5130a8cb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 34001 } }
I20260809 04:47:46.453328 31202 leader_election.cc:288] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4f0bfd4458e4cbd829e905adf720366 (127.12.234.2:36001), f2de2489efa24bc591f6a2963c41d6f8 (127.12.234.3:36117)
I20260809 04:47:46.459760 31197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:46.453617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:48459 (local address 127.12.234.2:36001)
0809 04:47:46.453807 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:46.453813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:46.454661 (+ 848us) server_negotiation.cc:407] Connection header received
0809 04:47:46.454878 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:46.454885 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:46.454954 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:46.455024 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:46.455677 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.456522 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.458215 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.458929 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.459086 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:46.459154 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:46.459415 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 04:47:46.459533 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:47:46.459964 31213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:46.453645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:36001 (local address 127.12.234.1:48459)
0809 04:47:46.454546 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:46.454562 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:46.454687 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:46.455080 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:46.455088 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:46.455115 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:46.455511 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:46.455528 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.456666 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.456671 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:46.458061 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:46.458073 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.459083 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.459089 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:46.459251 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:46.459289 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 04:47:46.459394 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 04:47:46.459506 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"mutex_wait_us":41,"thread_start_us":644,"threads_started":1}
I20260809 04:47:46.460055 31199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:46.453842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:60205 (local address 127.12.234.3:36117)
0809 04:47:46.454093 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:46.454102 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:46.454771 (+ 669us) server_negotiation.cc:407] Connection header received
0809 04:47:46.454933 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:46.454937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:46.455013 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:46.455107 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:47:46.456027 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.456824 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.458378 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.459094 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.459261 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:46.459325 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:46.459659 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 04:47:46.459828 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 04:47:46.460027 31211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:46.453690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:36117 (local address 127.12.234.1:60205)
0809 04:47:46.454642 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0809 04:47:46.454662 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:47:46.454776 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:47:46.455412 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:47:46.455415 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:47:46.455434 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:47:46.455847 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:46.455861 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.456954 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.456959 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:46.458240 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:47:46.458251 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.459283 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.459289 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:47:46.459430 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:46.459450 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 04:47:46.459599 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 04:47:46.459714 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":826,"thread_start_us":448,"threads_started":1}
I20260809 04:47:46.461023 30981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ee831722a22446db27a55e905d9f05f" candidate_uuid: "64c3ecf36c9a436b8893781f5130a8cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4f0bfd4458e4cbd829e905adf720366" is_pre_election: true
I20260809 04:47:46.461259 31129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ee831722a22446db27a55e905d9f05f" candidate_uuid: "64c3ecf36c9a436b8893781f5130a8cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2de2489efa24bc591f6a2963c41d6f8" is_pre_election: true
I20260809 04:47:46.461401 30981 raft_consensus.cc:2454] T 8ee831722a22446db27a55e905d9f05f P a4f0bfd4458e4cbd829e905adf720366 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64c3ecf36c9a436b8893781f5130a8cb in term 0.
I20260809 04:47:46.461651 31129 raft_consensus.cc:2454] T 8ee831722a22446db27a55e905d9f05f P f2de2489efa24bc591f6a2963c41d6f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 64c3ecf36c9a436b8893781f5130a8cb in term 0.
I20260809 04:47:46.461966 30749 leader_election.cc:302] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64c3ecf36c9a436b8893781f5130a8cb, a4f0bfd4458e4cbd829e905adf720366; no voters:
I20260809 04:47:46.462308 31202 raft_consensus.cc:2781] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:47:46.462455 31202 raft_consensus.cc:482] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:47:46.462551 31202 raft_consensus.cc:3037] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:47:46.464592 31202 raft_consensus.cc:504] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f0bfd4458e4cbd829e905adf720366" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 36001 } } peers { permanent_uuid: "f2de2489efa24bc591f6a2963c41d6f8" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 36117 } } peers { permanent_uuid: "64c3ecf36c9a436b8893781f5130a8cb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 34001 } }
I20260809 04:47:46.465150 31202 leader_election.cc:288] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb [CANDIDATE]: Term 1 election: Requested vote from peers a4f0bfd4458e4cbd829e905adf720366 (127.12.234.2:36001), f2de2489efa24bc591f6a2963c41d6f8 (127.12.234.3:36117)
I20260809 04:47:46.465485 30981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ee831722a22446db27a55e905d9f05f" candidate_uuid: "64c3ecf36c9a436b8893781f5130a8cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4f0bfd4458e4cbd829e905adf720366"
I20260809 04:47:46.465620 31129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ee831722a22446db27a55e905d9f05f" candidate_uuid: "64c3ecf36c9a436b8893781f5130a8cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2de2489efa24bc591f6a2963c41d6f8"
I20260809 04:47:46.465669 30981 raft_consensus.cc:3037] T 8ee831722a22446db27a55e905d9f05f P a4f0bfd4458e4cbd829e905adf720366 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:47:46.465811 31129 raft_consensus.cc:3037] T 8ee831722a22446db27a55e905d9f05f P f2de2489efa24bc591f6a2963c41d6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:47:46.470810 30981 raft_consensus.cc:2454] T 8ee831722a22446db27a55e905d9f05f P a4f0bfd4458e4cbd829e905adf720366 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64c3ecf36c9a436b8893781f5130a8cb in term 1.
I20260809 04:47:46.470810 31129 raft_consensus.cc:2454] T 8ee831722a22446db27a55e905d9f05f P f2de2489efa24bc591f6a2963c41d6f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 64c3ecf36c9a436b8893781f5130a8cb in term 1.
I20260809 04:47:46.471269 30749 leader_election.cc:302] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 64c3ecf36c9a436b8893781f5130a8cb, a4f0bfd4458e4cbd829e905adf720366, f2de2489efa24bc591f6a2963c41d6f8; no voters:
I20260809 04:47:46.471586 31202 raft_consensus.cc:2781] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:47:46.472038 31202 raft_consensus.cc:686] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb [term 1 LEADER]: Becoming Leader. State: Replica: 64c3ecf36c9a436b8893781f5130a8cb, State: Running, Role: LEADER
I20260809 04:47:46.472432 31202 consensus_queue.cc:237] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f0bfd4458e4cbd829e905adf720366" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 36001 } } peers { permanent_uuid: "f2de2489efa24bc591f6a2963c41d6f8" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 36117 } } peers { permanent_uuid: "64c3ecf36c9a436b8893781f5130a8cb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 34001 } }
I20260809 04:47:46.476680 30662 catalog_manager.cc:5495] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 64c3ecf36c9a436b8893781f5130a8cb (127.12.234.1). New cstate: current_term: 1 leader_uuid: "64c3ecf36c9a436b8893781f5130a8cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f0bfd4458e4cbd829e905adf720366" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 36001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2de2489efa24bc591f6a2963c41d6f8" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 36117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64c3ecf36c9a436b8893781f5130a8cb" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 34001 } health_report { overall_health: HEALTHY } } }
I20260809 04:47:46.496801 31218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:46.491639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32850 (local address 127.12.234.1:34001)
0809 04:47:46.492199 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:46.492203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:46.492221 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:47:46.492290 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:46.492293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:46.492365 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:46.492481 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:47:46.492809 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.493488 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.494816 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.495212 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.495296 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:46.495718 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:47:46.496255 (+ 537us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:47:46.496353 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:46.496501 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 04:47:46.496571 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":300,"threads_started":1}
I20260809 04:47:46.502256 31129 raft_consensus.cc:1261] T 8ee831722a22446db27a55e905d9f05f P f2de2489efa24bc591f6a2963c41d6f8 [term 1 FOLLOWER]: Refusing update from remote peer 64c3ecf36c9a436b8893781f5130a8cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:47:46.502251 30981 raft_consensus.cc:1261] T 8ee831722a22446db27a55e905d9f05f P a4f0bfd4458e4cbd829e905adf720366 [term 1 FOLLOWER]: Refusing update from remote peer 64c3ecf36c9a436b8893781f5130a8cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:47:46.503290 31215 consensus_queue.cc:1035] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4f0bfd4458e4cbd829e905adf720366" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 36001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:47:46.503646 31202 consensus_queue.cc:1035] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2de2489efa24bc591f6a2963c41d6f8" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 36117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:47:46.514063 31219 mvcc.cc:205] Tried to move back new op lower bound from 7316483549184057344 to 7316483549074599936. Current Snapshot: MvccSnapshot[applied={T|T < 7316483549184057344}]
I20260809 04:47:46.519192 31220 mvcc.cc:205] Tried to move back new op lower bound from 7316483549184057344 to 7316483549074599936. Current Snapshot: MvccSnapshot[applied={T|T < 7316483549184057344 or (T in {7316483549184057344})}]
I20260809 04:47:46.525700 30663 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:47:46.525933 30663 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:47:46.529027 30663 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98: Sending DeleteTablet for 3 replicas of tablet 8ee831722a22446db27a55e905d9f05f
I20260809 04:47:46.529918 31107 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ee831722a22446db27a55e905d9f05f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:47:46 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:40644
I20260809 04:47:46.529896 30960 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ee831722a22446db27a55e905d9f05f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:47:46 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:38934
I20260809 04:47:46.530332 30804 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ee831722a22446db27a55e905d9f05f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:47:46 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:32846
I20260809 04:47:46.531041 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 30705
I20260809 04:47:46.541497 31232 tablet_replica.cc:330] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb: stopping tablet replica
I20260809 04:47:46.541780 31232 raft_consensus.cc:2229] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:47:46.542389 31232 raft_consensus.cc:2258] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:47:46.543466 31232 ts_tablet_manager.cc:1905] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:47:46.544620 31230 tablet_replica.cc:330] T 8ee831722a22446db27a55e905d9f05f P a4f0bfd4458e4cbd829e905adf720366: stopping tablet replica
I20260809 04:47:46.544978 31231 tablet_replica.cc:330] T 8ee831722a22446db27a55e905d9f05f P f2de2489efa24bc591f6a2963c41d6f8: stopping tablet replica
I20260809 04:47:46.545477 31231 raft_consensus.cc:2229] T 8ee831722a22446db27a55e905d9f05f P f2de2489efa24bc591f6a2963c41d6f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:47:46.545960 31230 raft_consensus.cc:2229] T 8ee831722a22446db27a55e905d9f05f P a4f0bfd4458e4cbd829e905adf720366 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:47:46.546478 31230 raft_consensus.cc:2258] T 8ee831722a22446db27a55e905d9f05f P a4f0bfd4458e4cbd829e905adf720366 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:47:46.546907 31231 raft_consensus.cc:2258] T 8ee831722a22446db27a55e905d9f05f P f2de2489efa24bc591f6a2963c41d6f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:47:46.549161 31231 ts_tablet_manager.cc:1905] T 8ee831722a22446db27a55e905d9f05f P f2de2489efa24bc591f6a2963c41d6f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:47:46.550439 31230 ts_tablet_manager.cc:1905] T 8ee831722a22446db27a55e905d9f05f P a4f0bfd4458e4cbd829e905adf720366: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:47:46.550724 31232 ts_tablet_manager.cc:1918] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:47:46.550848 31232 log.cc:1196] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/wal/wals/8ee831722a22446db27a55e905d9f05f
I20260809 04:47:46.551311 31232 ts_tablet_manager.cc:1939] T 8ee831722a22446db27a55e905d9f05f P 64c3ecf36c9a436b8893781f5130a8cb: Deleting consensus metadata
I20260809 04:47:46.552577 30643 catalog_manager.cc:4841] TS 64c3ecf36c9a436b8893781f5130a8cb (127.12.234.1:34001): tablet 8ee831722a22446db27a55e905d9f05f (table test-table [id=6f9db36101a34389acbf0954ebc35e46]) successfully deleted
I20260809 04:47:46.554085 31218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:46.532428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32858 (local address 127.12.234.1:34001)
0809 04:47:46.532635 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:46.532642 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:46.532664 (+ 22us) server_negotiation.cc:407] Connection header received
0809 04:47:46.532722 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:46.532729 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:46.532815 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:46.532940 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:46.535477 (+ 2537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.536851 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.539194 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.540138 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.540275 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:46.546922 (+ 6647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:46.546930 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:46.546934 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:46.546971 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:46.550872 (+ 3901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:46.551766 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:46.551771 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:46.551774 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:46.551938 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:46.552465 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:46.552469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:46.552473 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:46.552835 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:46.553064 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:46.553468 (+ 404us) server_negotiation.cc:299] Negotiation successful
0809 04:47:46.553717 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 04:47:46.554739 30871 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:47:46.559625 31230 ts_tablet_manager.cc:1918] T 8ee831722a22446db27a55e905d9f05f P a4f0bfd4458e4cbd829e905adf720366: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:47:46.559767 31230 log.cc:1196] T 8ee831722a22446db27a55e905d9f05f P a4f0bfd4458e4cbd829e905adf720366: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/wal/wals/8ee831722a22446db27a55e905d9f05f
I20260809 04:47:46.560333 31230 ts_tablet_manager.cc:1939] T 8ee831722a22446db27a55e905d9f05f P a4f0bfd4458e4cbd829e905adf720366: Deleting consensus metadata
I20260809 04:47:46.560973 31231 ts_tablet_manager.cc:1918] T 8ee831722a22446db27a55e905d9f05f P f2de2489efa24bc591f6a2963c41d6f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:47:46.561122 31231 log.cc:1196] T 8ee831722a22446db27a55e905d9f05f P f2de2489efa24bc591f6a2963c41d6f8: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/wal/wals/8ee831722a22446db27a55e905d9f05f
I20260809 04:47:46.561690 31231 ts_tablet_manager.cc:1939] T 8ee831722a22446db27a55e905d9f05f P f2de2489efa24bc591f6a2963c41d6f8: Deleting consensus metadata
I20260809 04:47:46.562144 30645 catalog_manager.cc:4841] TS a4f0bfd4458e4cbd829e905adf720366 (127.12.234.2:36001): tablet 8ee831722a22446db27a55e905d9f05f (table test-table [id=6f9db36101a34389acbf0954ebc35e46]) successfully deleted
I20260809 04:47:46.563056 30641 catalog_manager.cc:4841] TS f2de2489efa24bc591f6a2963c41d6f8 (127.12.234.3:36117): tablet 8ee831722a22446db27a55e905d9f05f (table test-table [id=6f9db36101a34389acbf0954ebc35e46]) successfully deleted
I20260809 04:47:46.647630 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 30705
I20260809 04:47:46.670435 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 30884
I20260809 04:47:46.683169 31197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:46.671165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38938 (local address 127.12.234.2:36001)
0809 04:47:46.671336 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:46.671341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:46.671358 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:47:46.671498 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:46.671501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:46.671580 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:46.671656 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:46.672988 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.673711 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.674964 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.675366 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.675451 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:46.678122 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:46.678134 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:46.678148 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:46.678178 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:46.680857 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:46.681450 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:46.681456 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:46.681458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:46.681566 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:46.681854 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:46.681857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:46.681858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:46.682202 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:46.682431 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:46.682675 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 04:47:46.682819 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 04:47:46.683784 31027 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:47:46.765704 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 30884
I20260809 04:47:46.786041 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 31037
I20260809 04:47:46.798869 31199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:47:46.786699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40646 (local address 127.12.234.3:36117)
0809 04:47:46.786865 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 04:47:46.786870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:47:46.786887 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:47:46.786967 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:47:46.786971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:47:46.787027 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:47:46.787118 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:47:46.788235 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.789023 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.790487 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:47:46.790972 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:47:46.791080 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:47:46.793641 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:47:46.793654 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:47:46.793672 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:47:46.793701 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:47:46.796313 (+ 2612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:46.797050 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:46.797057 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:46.797059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:46.797234 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:47:46.797557 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:47:46.797560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:47:46.797562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:47:46.797942 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:47:46.798167 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 04:47:46.798460 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 04:47:46.798628 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:47:46.799477 31174 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:47:46.882643 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 31037
I20260809 04:47:46.903294 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 30623
I20260809 04:47:46.904418 30698 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:47:47.012120 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 30623
I20260809 04:48:07.039439 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:40967
--webserver_interface=127.12.234.62
--webserver_port=38989
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:42447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:40967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:48:07.373021 31238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:07.373426 31238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:07.373588 31238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:07.381107 31238 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 04:48:07.381194 31238 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 04:48:07.381274 31238 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 04:48:07.381369 31238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:48:07.381458 31238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:07.381556 31238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:07.381626 31238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:48:07.381699 31238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:48:07.391444 31238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:42447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:40967
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:40967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=38989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:07.392743 31238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:07.394054 31238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:07.406116 31245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:07.406152 31246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:07.406311 31248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:07.406934 31238 server_base.cc:1034] running on GCE node
I20260809 04:48:07.407729 31238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:07.409117 31238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:07.410372 31238 hybrid_clock.cc:648] HybridClock initialized: now 1786250887410307 us; error 59 us; skew 500 ppm
I20260809 04:48:07.410681 31238 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:48:07 dist-test-slave-1jjb krb5kdc[29992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250887, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:07.413583 31238 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:48:07.414870 31238 webserver.cc:460] Webserver started at http://127.12.234.62:38989/ using document root <none> and password file <none>
I20260809 04:48:07.415479 31238 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:07.415573 31238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:07.421023 31238 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:48:07.424177 31256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:07.425550 31238 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:48:07.425704 31238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "5695b9eea47c429d98e81934ebd75f98"
format_stamp: "Formatted at 2026-08-09 04:47:21 on dist-test-slave-1jjb"
server_key: "195c367a48003c7baa84a860dca6a8e5"
server_key_iv: "e11c3bafb2da7448cbdbde76601618b0"
server_key_version: "encryptionkey@0"
I20260809 04:48:07.426693 31238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:07.449514 31238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:07.453648 31238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:07.453907 31238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:07.475248 31238 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:40967
I20260809 04:48:07.475272 31320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:40967 every 8 connection(s)
I20260809 04:48:07.477186 31238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:48:07.479573 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 31238
I20260809 04:48:07.480479 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:34001
--local_ip_for_outbound_sockets=127.12.234.1
--tserver_master_addrs=127.12.234.62:40967
--webserver_port=40791
--webserver_interface=127.12.234.1
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:42447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:48:07.483935 31323 sys_catalog.cc:263] Verifying existing consensus state
I20260809 04:48:07.487156 31323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98: Bootstrap starting.
I20260809 04:48:07.514037 31323 log.cc:824] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:07.535589 31323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 04:48:07.536334 31323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98: Bootstrap complete.
I20260809 04:48:07.550365 31323 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695b9eea47c429d98e81934ebd75f98" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 40967 } }
I20260809 04:48:07.551628 31323 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5695b9eea47c429d98e81934ebd75f98, State: Initialized, Role: FOLLOWER
I20260809 04:48:07.552342 31323 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695b9eea47c429d98e81934ebd75f98" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 40967 } }
I20260809 04:48:07.552623 31323 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:48:07.552747 31323 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:48:07.552951 31323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [term 2 FOLLOWER]: Advancing to term 3
I20260809 04:48:07.559332 31323 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695b9eea47c429d98e81934ebd75f98" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 40967 } }
I20260809 04:48:07.559989 31323 leader_election.cc:302] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5695b9eea47c429d98e81934ebd75f98; no voters:
I20260809 04:48:07.560673 31323 leader_election.cc:288] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 04:48:07.560823 31327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 04:48:07.562233 31327 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [term 3 LEADER]: Becoming Leader. State: Replica: 5695b9eea47c429d98e81934ebd75f98, State: Running, Role: LEADER
I20260809 04:48:07.562776 31323 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:48:07.562747 31327 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695b9eea47c429d98e81934ebd75f98" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 40967 } }
I20260809 04:48:07.565140 31330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5695b9eea47c429d98e81934ebd75f98. Latest consensus state: current_term: 3 leader_uuid: "5695b9eea47c429d98e81934ebd75f98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695b9eea47c429d98e81934ebd75f98" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 40967 } } }
I20260809 04:48:07.565497 31330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:07.566084 31328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "5695b9eea47c429d98e81934ebd75f98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5695b9eea47c429d98e81934ebd75f98" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 40967 } } }
I20260809 04:48:07.566430 31328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98 [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:07.576602 31335 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:48:07.585659 31335 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:48:07.587199 31335 catalog_manager.cc:1259] Loaded cluster ID: 011cef416d964908a136759b2817a2ae
I20260809 04:48:07.587343 31335 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:48:07.591542 31335 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:48:07.593210 31335 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98: Loaded TSK: 0
I20260809 04:48:07.593654 31335 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5695b9eea47c429d98e81934ebd75f98: Loaded TSK: 1
F20260809 04:48:07.594825 31335 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f9da097f9ad google::LogMessage::Fail() at ??:0
@ 0x7f9da098288a google::LogMessage::SendToLog() at ??:0
@ 0x7f9da097f4de google::LogMessage::Flush() at ??:0
@ 0x7f9da0980a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9da46e37d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f9da46d273e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f9db63ae1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f9db63ad783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f9db64392a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f9db64390c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f9db64835e9 std::function<>::operator()() at ??:0
@ 0x7f9db63b1cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f9db63afc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9db6437610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f9db6437439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f9da35cb345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f9da35ce91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f9da35ce799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f9da35a9eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f9da68836db start_thread at ??:0
@ 0x7f9d9f81961f clone at ??:0
W20260809 04:48:07.894363 31325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:07.894912 31325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:07.895128 31325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:07.902630 31325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:07.902742 31325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:07.902872 31325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:48:07.912959 31325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:42447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:34001
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=40791
--tserver_master_addrs=127.12.234.62:40967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:07.914248 31325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:07.915527 31325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:07.929397 31376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:07.929486 31377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:07.929512 31379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:07.933727 31325 server_base.cc:1034] running on GCE node
I20260809 04:48:07.934418 31325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:07.935351 31325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:07.936547 31325 hybrid_clock.cc:648] HybridClock initialized: now 1786250887936497 us; error 38 us; skew 500 ppm
I20260809 04:48:07.936874 31325 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:48:07 dist-test-slave-1jjb krb5kdc[29992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250887, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:07.939982 31325 init.cc:377] Logged in from keytab as kudu/127.12.234.1@KRBTEST.COM (short username kudu)
I20260809 04:48:07.941593 31325 webserver.cc:460] Webserver started at http://127.12.234.1:40791/ using document root <none> and password file <none>
I20260809 04:48:07.942286 31325 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:07.942418 31325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:07.947739 31325 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:48:07.950356 31386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:07.951730 31325 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 04:48:07.951922 31325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "64c3ecf36c9a436b8893781f5130a8cb"
format_stamp: "Formatted at 2026-08-09 04:47:22 on dist-test-slave-1jjb"
server_key: "553137d64b696ca9eb85cd1d45699e12"
server_key_iv: "3937e74f3c5c6686ec9611523cbe2752"
server_key_version: "encryptionkey@0"
I20260809 04:48:07.952978 31325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:07.977393 31325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:07.981860 31325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:07.982178 31325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:07.983342 31325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:07.984938 31325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:48:07.985082 31325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:07.985212 31325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:07.985311 31325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:08.020231 31325 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:34001
I20260809 04:48:08.020283 31511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:34001 every 8 connection(s)
I20260809 04:48:08.021658 31325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:48:08.026427 31514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:08.024623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.1:59287)
0809 04:48:08.025340 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:08.025590 (+ 250us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":278,"threads_started":1}
W20260809 04:48:08.027081 31512 heartbeater.cc:644] Failed to heartbeat to 127.12.234.62:40967 (0 consecutive failures): Network error: Failed to ping master at 127.12.234.62:40967: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
I20260809 04:48:08.027846 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 31325
I20260809 04:48:08.027886 31514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:08.027594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.1:47411)
0809 04:48:08.027727 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:08.027786 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 04:48:08.028693 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:36001
--local_ip_for_outbound_sockets=127.12.234.2
--tserver_master_addrs=127.12.234.62:40967
--webserver_port=34997
--webserver_interface=127.12.234.2
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:42447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:48:08.028868 31514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:08.028511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.1:42509)
0809 04:48:08.028639 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:08.028731 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 04:48:08.029111 31512 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 04:48:08.358894 31517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:08.359153 31517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:08.359243 31517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:08.366034 31517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:08.366130 31517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:08.366235 31517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:48:08.375236 31517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:42447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:36001
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=34997
--tserver_master_addrs=127.12.234.62:40967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:08.376901 31517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:08.378415 31517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:08.391075 31523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:08.391049 31524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:08.391335 31526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:08.392473 31517 server_base.cc:1034] running on GCE node
I20260809 04:48:08.393113 31517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:08.394021 31517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:08.395222 31517 hybrid_clock.cc:648] HybridClock initialized: now 1786250888395190 us; error 46 us; skew 500 ppm
I20260809 04:48:08.395494 31517 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:48:08 dist-test-slave-1jjb krb5kdc[29992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250888, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:08.398488 31517 init.cc:377] Logged in from keytab as kudu/127.12.234.2@KRBTEST.COM (short username kudu)
I20260809 04:48:08.399753 31517 webserver.cc:460] Webserver started at http://127.12.234.2:34997/ using document root <none> and password file <none>
I20260809 04:48:08.400460 31517 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:08.400600 31517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:08.405548 31517 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260809 04:48:08.408252 31534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:08.409462 31517 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 04:48:08.409632 31517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "a4f0bfd4458e4cbd829e905adf720366"
format_stamp: "Formatted at 2026-08-09 04:47:22 on dist-test-slave-1jjb"
server_key: "9261fad4449bd10e24901bcb0a3b1955"
server_key_iv: "10ca47b25d37ebc037d9d5468c7dc75e"
server_key_version: "encryptionkey@0"
I20260809 04:48:08.410588 31517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:08.433534 31517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:08.437469 31517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:08.437754 31517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:08.438933 31517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:08.440618 31517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:48:08.440749 31517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:08.440883 31517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:08.440968 31517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:08.474205 31517 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:36001
I20260809 04:48:08.474282 31660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:36001 every 8 connection(s)
I20260809 04:48:08.475544 31517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:48:08.477231 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 31517
I20260809 04:48:08.478010 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:36117
--local_ip_for_outbound_sockets=127.12.234.3
--tserver_master_addrs=127.12.234.62:40967
--webserver_port=33145
--webserver_interface=127.12.234.3
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:42447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:48:08.480913 31665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:08.478781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.2:36455)
0809 04:48:08.479567 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:08.479793 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":333,"threads_started":1}
W20260809 04:48:08.481608 31662 heartbeater.cc:644] Failed to heartbeat to 127.12.234.62:40967 (0 consecutive failures): Network error: Failed to ping master at 127.12.234.62:40967: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
I20260809 04:48:08.482410 31665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:08.482095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.2:40925)
0809 04:48:08.482239 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:08.482311 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 04:48:08.483336 31665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:08.482894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.2:46511)
0809 04:48:08.483066 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:08.483165 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 04:48:08.483560 31662 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 04:48:08.807365 31666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:08.807641 31666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:08.807739 31666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:08.815230 31666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:08.815346 31666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:08.815464 31666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:48:08.825052 31666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:42447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:36117
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=33145
--tserver_master_addrs=127.12.234.62:40967
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:08.826297 31666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:08.827557 31666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:08.840351 31675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:08.840454 31673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:08.840682 31671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:08.841224 31666 server_base.cc:1034] running on GCE node
I20260809 04:48:08.841853 31666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:08.842747 31666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:08.843974 31666 hybrid_clock.cc:648] HybridClock initialized: now 1786250888843939 us; error 44 us; skew 500 ppm
I20260809 04:48:08.844303 31666 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:48:08 dist-test-slave-1jjb krb5kdc[29992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250888, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:08.847344 31666 init.cc:377] Logged in from keytab as kudu/127.12.234.3@KRBTEST.COM (short username kudu)
I20260809 04:48:08.848804 31666 webserver.cc:460] Webserver started at http://127.12.234.3:33145/ using document root <none> and password file <none>
I20260809 04:48:08.849570 31666 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:08.849723 31666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:08.854846 31666 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 04:48:08.857582 31682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:08.858750 31666 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 04:48:08.858914 31666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "f2de2489efa24bc591f6a2963c41d6f8"
format_stamp: "Formatted at 2026-08-09 04:47:23 on dist-test-slave-1jjb"
server_key: "fc7dca100bf153b7bad0f2f4a5ab64da"
server_key_iv: "2edc1153d99e56082098e456cbe1c05a"
server_key_version: "encryptionkey@0"
I20260809 04:48:08.859793 31666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:08.878046 31666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:08.881703 31666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:08.882009 31666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:08.882973 31666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:08.884279 31666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:48:08.884382 31666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:08.884502 31666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:08.884588 31666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:08.920765 31666 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:36117
I20260809 04:48:08.920841 31805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:36117 every 8 connection(s)
I20260809 04:48:08.922148 31666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:48:08.926578 31808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:08.924874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.3:39263)
0809 04:48:08.925546 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:08.925767 (+ 221us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":229,"threads_started":1}
I20260809 04:48:08.926990 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 31666
W20260809 04:48:08.927282 31806 heartbeater.cc:644] Failed to heartbeat to 127.12.234.62:40967 (0 consecutive failures): Network error: Failed to ping master at 127.12.234.62:40967: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
I20260809 04:48:08.928170 31808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:08.927769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.3:41955)
0809 04:48:08.927921 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:08.928009 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 04:48:08.929136 31808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:08.928809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.3:43833)
0809 04:48:08.928929 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:08.929016 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 04:48:08.929415 31806 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 04:48:09.030342 31810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:09.029598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.1:40089)
0809 04:48:09.030024 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:09.030117 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":199,"threads_started":1}
I20260809 04:48:09.484910 31811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:09.484127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.2:60387)
0809 04:48:09.484615 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:09.484694 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":258,"threads_started":1}
I20260809 04:48:09.930619 31812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:09.929880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.3:49525)
0809 04:48:09.930315 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:09.930421 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":225,"threads_started":1}
I20260809 04:48:10.031960 31813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:10.031175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.1:34285)
0809 04:48:10.031682 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:10.031765 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":278,"threads_started":1}
I20260809 04:48:10.486649 31814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:10.485844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.2:35611)
0809 04:48:10.486381 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:10.486465 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":258,"threads_started":1}
I20260809 04:48:10.932323 31815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:10.931512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.3:37247)
0809 04:48:10.932008 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:10.932168 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":272,"threads_started":1}
I20260809 04:48:11.033707 31816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:11.032980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.1:40707)
0809 04:48:11.033417 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:11.033532 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":231,"threads_started":1}
I20260809 04:48:11.488435 31818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:11.487680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.2:35293)
0809 04:48:11.488175 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:11.488260 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":229,"threads_started":1}
I20260809 04:48:11.933706 31819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:11.933075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.3:53707)
0809 04:48:11.933479 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:11.933553 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":217,"threads_started":1}
I20260809 04:48:12.035511 31820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:12.034687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.1:56209)
0809 04:48:12.035203 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:12.035293 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":249,"threads_started":1}
I20260809 04:48:12.489964 31821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:12.489269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.2:50651)
0809 04:48:12.489701 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:12.489778 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":207,"threads_started":1}
I20260809 04:48:12.935422 31822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:12.934640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.3:47497)
0809 04:48:12.935137 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:12.935221 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":259,"threads_started":1}
I20260809 04:48:13.037263 31823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:13.036392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.1:42209)
0809 04:48:13.036922 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:13.037038 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":281,"threads_started":1}
I20260809 04:48:13.491529 31826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:13.490817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.2:35205)
0809 04:48:13.491270 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:13.491341 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":233,"threads_started":1}
I20260809 04:48:13.937029 31827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:13.936318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.3:34713)
0809 04:48:13.936784 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:13.936860 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":249,"threads_started":1}
I20260809 04:48:14.038851 31828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:14.038161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.1:34003)
0809 04:48:14.038593 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:14.038679 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":229,"threads_started":1}
I20260809 04:48:14.493209 31829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:14.492400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.2:50869)
0809 04:48:14.492841 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:14.492986 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":226,"threads_started":1}
I20260809 04:48:14.938704 31831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:14.937922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.3:36129)
0809 04:48:14.938453 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:14.938543 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":300,"threads_started":1}
I20260809 04:48:15.040603 31832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:15.039728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.1:33743)
0809 04:48:15.040303 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:15.040388 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":235,"threads_started":1}
I20260809 04:48:15.494789 31833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:15.494044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.2:44721)
0809 04:48:15.494521 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:15.494595 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":186,"threads_started":1}
I20260809 04:48:15.940459 31834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:15.939613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.3:43055)
0809 04:48:15.940176 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:15.940265 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":202,"threads_started":1}
I20260809 04:48:16.042309 31835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:16.041546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.1:50655)
0809 04:48:16.042050 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:16.042136 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":268,"threads_started":1}
I20260809 04:48:16.496393 31836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:16.495665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.2:44647)
0809 04:48:16.496157 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:16.496236 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":216,"threads_started":1}
I20260809 04:48:16.941848 31837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:16.941228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.3:44315)
0809 04:48:16.941627 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:16.941700 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":217,"threads_started":1}
I20260809 04:48:17.043920 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:17.043196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.1:39823)
0809 04:48:17.043666 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:17.043746 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":231,"threads_started":1}
I20260809 04:48:17.497972 31840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:17.497244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.2:49637)
0809 04:48:17.497674 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:17.497752 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":200,"threads_started":1}
I20260809 04:48:17.944674 31841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:17.943201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.3:48933)
0809 04:48:17.944252 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:17.944392 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":788,"thread_start_us":438,"threads_started":1}
I20260809 04:48:18.045624 31842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:18.044851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.1:39047)
0809 04:48:18.045378 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:18.045470 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":316,"threads_started":1}
I20260809 04:48:18.499480 31843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:18.498798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.2:54181)
0809 04:48:18.499241 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:18.499317 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":235,"threads_started":1}
I20260809 04:48:18.946544 31844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:18.945776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.3:42175)
0809 04:48:18.946285 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:18.946376 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":292,"threads_started":1}
I20260809 04:48:19.047323 31845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:19.046591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.1:48313)
0809 04:48:19.047036 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:19.047133 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":224,"threads_started":1}
I20260809 04:48:19.501334 31846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:19.500499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.2:40507)
0809 04:48:19.500999 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:19.501099 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":279,"threads_started":1}
I20260809 04:48:19.948199 31848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:19.947439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.3:53005)
0809 04:48:19.947862 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:19.947947 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":223,"threads_started":1}
I20260809 04:48:20.049041 31849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:20.048310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.1:44727)
0809 04:48:20.048798 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:20.048883 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":259,"threads_started":1}
I20260809 04:48:20.502916 31850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:20.502248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.2:33021)
0809 04:48:20.502686 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:20.502763 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":248,"threads_started":1}
I20260809 04:48:20.949841 31851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:20.949176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.3:46023)
0809 04:48:20.949605 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:20.949687 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":215,"threads_started":1}
I20260809 04:48:21.050630 31852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:21.049909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.1:53471)
0809 04:48:21.050353 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:21.050441 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":216,"threads_started":1}
I20260809 04:48:21.504735 31853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:21.503901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.2:34915)
0809 04:48:21.504424 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:21.504535 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":299,"threads_started":1}
I20260809 04:48:21.951562 31854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:21.950715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.3:41623)
0809 04:48:21.951270 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:21.951366 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":272,"threads_started":1}
I20260809 04:48:22.052416 31855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:22.051539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.1:56457)
0809 04:48:22.052027 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:22.052165 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":235,"threads_started":1}
I20260809 04:48:22.506242 31856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:22.505561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.2:57891)
0809 04:48:22.505996 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:22.506072 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":196,"threads_started":1}
I20260809 04:48:22.954177 31857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:22.952859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.3:44547)
0809 04:48:22.953834 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:22.953923 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":858,"thread_start_us":717,"threads_started":1}
I20260809 04:48:23.054973 31858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:23.053712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.1:49421)
0809 04:48:23.054509 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:23.054688 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":379,"threads_started":1}
I20260809 04:48:23.508672 31859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:23.507602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.2:55175)
0809 04:48:23.508259 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:23.508414 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":360,"threads_started":1}
I20260809 04:48:23.928753 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 31325
I20260809 04:48:23.949609 31860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:23.929967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55002 (local address 127.12.234.1:34001)
0809 04:48:23.930697 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:23.930719 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:23.930770 (+ 51us) server_negotiation.cc:407] Connection header received
0809 04:48:23.932595 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:23.932605 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:23.933018 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:23.933327 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:23.934740 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:23.935873 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:23.937016 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:23.937646 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:23.937793 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:23.943814 (+ 6021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:23.943822 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:23.943836 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:23.943873 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:23.946796 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:23.947406 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:23.947412 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:23.947418 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:23.947575 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:23.947888 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:23.947891 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:23.947893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:23.948463 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:23.948691 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:23.948990 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 04:48:23.949314 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":352,"threads_started":1}
I20260809 04:48:23.950934 31507 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:23.955932 31862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:23.955212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:40967 (local address 127.12.234.3:35625)
0809 04:48:23.955652 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:23.955731 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.62:40967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":243,"threads_started":1}
I20260809 04:48:24.029934 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 31325
I20260809 04:48:24.052448 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 31517
I20260809 04:48:24.069231 31863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:24.053303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58562 (local address 127.12.234.2:36001)
0809 04:48:24.053816 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:24.053830 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:24.053866 (+ 36us) server_negotiation.cc:407] Connection header received
0809 04:48:24.055437 (+ 1571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:24.055445 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:24.055872 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:24.056237 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:24.057474 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:24.058678 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:24.059783 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:24.060582 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:24.060710 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:24.063481 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:24.063488 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:24.063500 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:24.063534 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:24.066307 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:24.066858 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:24.066864 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:24.066868 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:24.067022 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:24.067349 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:24.067352 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:24.067354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:24.067801 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:24.068034 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:24.068552 (+ 518us) server_negotiation.cc:299] Negotiation successful
0809 04:48:24.068950 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":233,"threads_started":1}
I20260809 04:48:24.070518 31656 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:24.148936 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 31517
I20260809 04:48:24.170825 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 31666
I20260809 04:48:24.187520 31865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:24.171722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47978 (local address 127.12.234.3:36117)
0809 04:48:24.172354 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:24.172368 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:24.172409 (+ 41us) server_negotiation.cc:407] Connection header received
0809 04:48:24.174080 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:24.174089 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:24.174490 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:24.174824 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:24.176319 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:24.177539 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:24.178694 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:24.179272 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:24.179408 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:24.181783 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:24.181791 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:24.181804 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:24.181839 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:24.184853 (+ 3014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:24.185403 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:24.185409 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:24.185414 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:24.185571 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:24.185856 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:24.185859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:24.185861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:24.186287 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:24.186546 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:24.186809 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 04:48:24.187216 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":292,"threads_started":1}
I20260809 04:48:24.188848 31801 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:24.266644 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 31666
2026-08-09T04:48:24Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64156 ms)
[----------] 30 tests from SecurityITest (300820 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:48:24 dist-test-slave-1jjb krb5kdc[31869](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:48:24 dist-test-slave-1jjb krb5kdc[31869](info): set up 2 sockets
Aug 09 04:48:24 dist-test-slave-1jjb krb5kdc[31869](info): commencing operation
krb5kdc: starting...
W20260809 04:48:25.376324 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:48:25 dist-test-slave-1jjb krb5kdc[31869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:48:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:48:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:48:25.653785 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:36803
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:33583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:36803
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:48:25.979773 31882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:25.980145 31882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:25.980329 31882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:25.988111 31882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:48:25.988263 31882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:25.988381 31882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:25.988487 31882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:48:25.988564 31882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:48:25.998435 31882 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:33583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:36803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:36803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:25.999675 31882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:26.000926 31882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:26.012670 31887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:26.012746 31890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:26.012760 31888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:26.013689 31882 server_base.cc:1034] running on GCE node
I20260809 04:48:26.014410 31882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:26.015762 31882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:26.016992 31882 hybrid_clock.cc:648] HybridClock initialized: now 1786250906016945 us; error 38 us; skew 500 ppm
I20260809 04:48:26.017338 31882 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:48:26 dist-test-slave-1jjb krb5kdc[31869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250906, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:26.020479 31882 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:48:26.021847 31882 webserver.cc:460] Webserver started at http://127.12.234.62:39023/ using document root <none> and password file <none>
I20260809 04:48:26.022526 31882 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:26.022675 31882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:26.022979 31882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:26.025942 31882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "543156f42da845989b08172ded5c5f95"
format_stamp: "Formatted at 2026-08-09 04:48:26 on dist-test-slave-1jjb"
server_key: "85a947a237ee3b2ba4cec73ebc2990be"
server_key_iv: "197c1f489b14cab6bc152a256ca88bdd"
server_key_version: "encryptionkey@0"
I20260809 04:48:26.026634 31882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "543156f42da845989b08172ded5c5f95"
format_stamp: "Formatted at 2026-08-09 04:48:26 on dist-test-slave-1jjb"
server_key: "85a947a237ee3b2ba4cec73ebc2990be"
server_key_iv: "197c1f489b14cab6bc152a256ca88bdd"
server_key_version: "encryptionkey@0"
I20260809 04:48:26.031518 31882 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 04:48:26.034863 31899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:26.035990 31882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:48:26.036175 31882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "543156f42da845989b08172ded5c5f95"
format_stamp: "Formatted at 2026-08-09 04:48:26 on dist-test-slave-1jjb"
server_key: "85a947a237ee3b2ba4cec73ebc2990be"
server_key_iv: "197c1f489b14cab6bc152a256ca88bdd"
server_key_version: "encryptionkey@0"
I20260809 04:48:26.036378 31882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:26.051182 31882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:26.055296 31882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:26.055564 31882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:26.073683 31882 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:36803
I20260809 04:48:26.073690 31957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:36803 every 8 connection(s)
I20260809 04:48:26.074898 31882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:48:26.076237 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 31882
I20260809 04:48:26.076464 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:48:26.077355 13224 external_mini_cluster.cc:1428] Setting key af836d881dc411018ee4ed149603ba94
I20260809 04:48:26.079480 31958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 04:48:26 dist-test-slave-1jjb krb5kdc[31869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:48:26.092700 31958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 543156f42da845989b08172ded5c5f95: Bootstrap starting.
I20260809 04:48:26.096053 31958 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 543156f42da845989b08172ded5c5f95: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:26.097260 31958 log.cc:824] T 00000000000000000000000000000000 P 543156f42da845989b08172ded5c5f95: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:26.100400 31958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 543156f42da845989b08172ded5c5f95: No bootstrap required, opened a new log
I20260809 04:48:26.100605 31962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:26.079322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44986 (local address 127.12.234.62:36803)
0809 04:48:26.080042 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:26.080055 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:26.080130 (+ 75us) server_negotiation.cc:407] Connection header received
0809 04:48:26.082346 (+ 2216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:26.082372 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:26.082854 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:26.083239 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:26.084509 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:26.086538 (+ 2029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:26.087982 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:26.089051 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:26.089288 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:26.093509 (+ 4221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:26.093516 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:26.093528 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:26.093557 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:26.096576 (+ 3019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:26.097313 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:26.097319 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:26.097324 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:26.097545 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:26.098026 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:26.098033 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:26.098041 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:26.098736 (+ 695us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:26.098948 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:26.099352 (+ 404us) server_negotiation.cc:299] Negotiation successful
0809 04:48:26.099604 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":315,"threads_started":1}
I20260809 04:48:26.107502 31958 raft_consensus.cc:348] T 00000000000000000000000000000000 P 543156f42da845989b08172ded5c5f95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "543156f42da845989b08172ded5c5f95" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36803 } }
I20260809 04:48:26.107827 31958 raft_consensus.cc:374] T 00000000000000000000000000000000 P 543156f42da845989b08172ded5c5f95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:26.107960 31958 raft_consensus.cc:729] T 00000000000000000000000000000000 P 543156f42da845989b08172ded5c5f95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 543156f42da845989b08172ded5c5f95, State: Initialized, Role: FOLLOWER
I20260809 04:48:26.108558 31958 consensus_queue.cc:260] T 00000000000000000000000000000000 P 543156f42da845989b08172ded5c5f95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "543156f42da845989b08172ded5c5f95" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36803 } }
I20260809 04:48:26.108783 31958 raft_consensus.cc:388] T 00000000000000000000000000000000 P 543156f42da845989b08172ded5c5f95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:48:26.108925 31958 raft_consensus.cc:482] T 00000000000000000000000000000000 P 543156f42da845989b08172ded5c5f95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:48:26.109094 31958 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 543156f42da845989b08172ded5c5f95 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:26.111205 31958 raft_consensus.cc:504] T 00000000000000000000000000000000 P 543156f42da845989b08172ded5c5f95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "543156f42da845989b08172ded5c5f95" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36803 } }
I20260809 04:48:26.111640 31958 leader_election.cc:302] T 00000000000000000000000000000000 P 543156f42da845989b08172ded5c5f95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 543156f42da845989b08172ded5c5f95; no voters:
I20260809 04:48:26.112411 31958 leader_election.cc:288] T 00000000000000000000000000000000 P 543156f42da845989b08172ded5c5f95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:48:26.112468 31964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 543156f42da845989b08172ded5c5f95 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:48:26.113219 31964 raft_consensus.cc:686] T 00000000000000000000000000000000 P 543156f42da845989b08172ded5c5f95 [term 1 LEADER]: Becoming Leader. State: Replica: 543156f42da845989b08172ded5c5f95, State: Running, Role: LEADER
I20260809 04:48:26.113734 31964 consensus_queue.cc:237] T 00000000000000000000000000000000 P 543156f42da845989b08172ded5c5f95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "543156f42da845989b08172ded5c5f95" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36803 } }
I20260809 04:48:26.114351 31958 sys_catalog.cc:564] T 00000000000000000000000000000000 P 543156f42da845989b08172ded5c5f95 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:48:26.116250 31965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 543156f42da845989b08172ded5c5f95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "543156f42da845989b08172ded5c5f95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "543156f42da845989b08172ded5c5f95" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36803 } } }
I20260809 04:48:26.116607 31965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 543156f42da845989b08172ded5c5f95 [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:26.116484 31966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 543156f42da845989b08172ded5c5f95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 543156f42da845989b08172ded5c5f95. Latest consensus state: current_term: 1 leader_uuid: "543156f42da845989b08172ded5c5f95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "543156f42da845989b08172ded5c5f95" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36803 } } }
I20260809 04:48:26.116727 31966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 543156f42da845989b08172ded5c5f95 [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:26.118991 31971 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:48:26.124120 31971 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:48:26.132194 31971 catalog_manager.cc:1347] Generated new cluster ID: f1d600c4431d4eabaab91271e16900a4
I20260809 04:48:26.132290 31971 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:48:26.160981 31971 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:48:26.162573 31971 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:48:26.169576 31971 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 543156f42da845989b08172ded5c5f95: Generated new TSK 0
I20260809 04:48:26.170279 31971 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
I20260809 04:48:26.250272 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:0
--local_ip_for_outbound_sockets=127.12.234.1
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:36803
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:33583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:48:26.569540 31989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:26.569845 31989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:26.570024 31989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:26.576442 31989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:26.576560 31989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:26.576707 31989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:48:26.585610 31989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:33583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:36803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:26.586786 31989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:26.588045 31989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:26.600740 31995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:26.600716 31996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:26.600689 31998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:26.601899 31989 server_base.cc:1034] running on GCE node
I20260809 04:48:26.602432 31989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:26.603288 31989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:26.604557 31989 hybrid_clock.cc:648] HybridClock initialized: now 1786250906604519 us; error 50 us; skew 500 ppm
I20260809 04:48:26.604882 31989 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:48:26 dist-test-slave-1jjb krb5kdc[31869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250906, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:26.607987 31989 init.cc:377] Logged in from keytab as kudu/127.12.234.1@KRBTEST.COM (short username kudu)
I20260809 04:48:26.609489 31989 webserver.cc:460] Webserver started at http://127.12.234.1:40141/ using document root <none> and password file <none>
I20260809 04:48:26.610155 31989 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:26.610301 31989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:26.610610 31989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:26.613360 31989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-0/data/instance:
uuid: "11c92879d2354be390e4362c076a267b"
format_stamp: "Formatted at 2026-08-09 04:48:26 on dist-test-slave-1jjb"
server_key: "9acea544c09de6cfd9344d2b6f5d5ba8"
server_key_iv: "dcd3aa8a9ed031385d33ab26781e00b9"
server_key_version: "encryptionkey@0"
I20260809 04:48:26.614238 31989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance:
uuid: "11c92879d2354be390e4362c076a267b"
format_stamp: "Formatted at 2026-08-09 04:48:26 on dist-test-slave-1jjb"
server_key: "9acea544c09de6cfd9344d2b6f5d5ba8"
server_key_iv: "dcd3aa8a9ed031385d33ab26781e00b9"
server_key_version: "encryptionkey@0"
I20260809 04:48:26.619565 31989 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 04:48:26.622859 32005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:26.624076 31989 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:48:26.624289 31989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "11c92879d2354be390e4362c076a267b"
format_stamp: "Formatted at 2026-08-09 04:48:26 on dist-test-slave-1jjb"
server_key: "9acea544c09de6cfd9344d2b6f5d5ba8"
server_key_iv: "dcd3aa8a9ed031385d33ab26781e00b9"
server_key_version: "encryptionkey@0"
I20260809 04:48:26.624475 31989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:26.650605 31989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:26.654068 31989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:26.654341 31989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:26.655256 31989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:26.656512 31989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:48:26.656608 31989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:26.656760 31989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:26.656827 31989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:26.688956 31989 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:36659
I20260809 04:48:26.689034 32126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:36659 every 8 connection(s)
I20260809 04:48:26.690212 31989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:48:26.695341 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 31989
I20260809 04:48:26.695559 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance
I20260809 04:48:26.696281 13224 external_mini_cluster.cc:1428] Setting key b0e48f6eeab7cce5f31e670145777182
Aug 09 04:48:26 dist-test-slave-1jjb krb5kdc[31869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250906, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:48:26.711417 32129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:26.692887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:46777 (local address 127.12.234.62:36803)
0809 04:48:26.693316 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:26.693320 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:26.694178 (+ 858us) server_negotiation.cc:407] Connection header received
0809 04:48:26.695968 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:26.695973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:26.696055 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:26.696169 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:26.697908 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:26.698946 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:26.700512 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:26.701204 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:26.701354 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:26.704635 (+ 3281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:26.704643 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:26.704650 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:26.704691 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:26.708005 (+ 3314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:26.708865 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:26.708873 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:26.708878 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:26.709019 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:26.709497 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:26.709504 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:26.709510 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:26.709875 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:26.710024 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:26.710738 (+ 714us) server_negotiation.cc:299] Negotiation successful
0809 04:48:26.710965 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":235,"threads_started":1}
I20260809 04:48:26.712214 32130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:26.693342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:36803 (local address 127.12.234.1:46777)
0809 04:48:26.694017 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:26.694064 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:26.695712 (+ 1648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:26.696461 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:26.696468 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:26.697010 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:48:26.697711 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:26.697735 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:26.699126 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:26.699130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:26.700378 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:26.700389 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:26.701412 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:26.701417 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:26.701586 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:26.702215 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:48:26.702239 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:48:26.704417 (+ 2178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:48:26.708167 (+ 3750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:26.708173 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:26.708184 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:26.708747 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:26.709160 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:26.709166 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:26.709172 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:26.709388 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:26.709998 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:48:26.710007 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:48:26.710379 (+ 372us) client_negotiation.cc:769] Sending connection context
0809 04:48:26.710618 (+ 239us) client_negotiation.cc:240] Negotiation successful
0809 04:48:26.711047 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":267,"threads_started":1}
I20260809 04:48:26.713677 32127 heartbeater.cc:347] Connected to a master server at 127.12.234.62:36803
I20260809 04:48:26.713944 32127 heartbeater.cc:464] Registering TS with master...
I20260809 04:48:26.714697 32127 heartbeater.cc:511] Master 127.12.234.62:36803 requested a full tablet report, sending...
I20260809 04:48:26.716885 31917 ts_manager.cc:194] Registered new tserver with Master: 11c92879d2354be390e4362c076a267b (127.12.234.1:36659)
I20260809 04:48:26.719023 31917 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.1@KRBTEST.COM'} at 127.12.234.1:46777
WARNING: no policy specified for kudu/127.12.234.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
I20260809 04:48:26.763661 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:0
--local_ip_for_outbound_sockets=127.12.234.2
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:36803
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:33583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:48:27.083868 32133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:27.084179 32133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:27.084345 32133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:27.091298 32133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:27.091449 32133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:27.091626 32133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:48:27.101109 32133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:33583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:36803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:27.102304 32133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:27.103538 32133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:27.116374 32139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:27.116340 32140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:27.116546 32142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:27.121376 32133 server_base.cc:1034] running on GCE node
I20260809 04:48:27.121948 32133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:27.122816 32133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:27.123999 32133 hybrid_clock.cc:648] HybridClock initialized: now 1786250907123964 us; error 24 us; skew 500 ppm
I20260809 04:48:27.124362 32133 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:48:27 dist-test-slave-1jjb krb5kdc[31869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250907, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:27.127444 32133 init.cc:377] Logged in from keytab as kudu/127.12.234.2@KRBTEST.COM (short username kudu)
I20260809 04:48:27.128993 32133 webserver.cc:460] Webserver started at http://127.12.234.2:41839/ using document root <none> and password file <none>
I20260809 04:48:27.129670 32133 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:27.129815 32133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:27.130121 32133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:27.132813 32133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-1/data/instance:
uuid: "8a044a6257334b8c90423d8cc75bf54b"
format_stamp: "Formatted at 2026-08-09 04:48:27 on dist-test-slave-1jjb"
server_key: "a83b052630df3496d821a8c5d24c55ba"
server_key_iv: "1fe954e0277f2b55861a195ab6991bdf"
server_key_version: "encryptionkey@0"
I20260809 04:48:27.133543 32133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance:
uuid: "8a044a6257334b8c90423d8cc75bf54b"
format_stamp: "Formatted at 2026-08-09 04:48:27 on dist-test-slave-1jjb"
server_key: "a83b052630df3496d821a8c5d24c55ba"
server_key_iv: "1fe954e0277f2b55861a195ab6991bdf"
server_key_version: "encryptionkey@0"
I20260809 04:48:27.138195 32133 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:48:27.141284 32149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:27.142457 32133 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:48:27.142630 32133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "8a044a6257334b8c90423d8cc75bf54b"
format_stamp: "Formatted at 2026-08-09 04:48:27 on dist-test-slave-1jjb"
server_key: "a83b052630df3496d821a8c5d24c55ba"
server_key_iv: "1fe954e0277f2b55861a195ab6991bdf"
server_key_version: "encryptionkey@0"
I20260809 04:48:27.142807 32133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:27.177852 32133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:27.181571 32133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:27.181840 32133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:27.182739 32133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:27.183993 32133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:48:27.184126 32133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:27.184270 32133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:27.184341 32133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:27.217182 32133 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:38249
I20260809 04:48:27.217263 32271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:38249 every 8 connection(s)
I20260809 04:48:27.218400 32133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:48:27.220490 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 32133
I20260809 04:48:27.220676 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance
I20260809 04:48:27.221196 13224 external_mini_cluster.cc:1428] Setting key 82112f0c1af51ebcf20b82eff8667f90
Aug 09 04:48:27 dist-test-slave-1jjb krb5kdc[31869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250907, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:48:27.238332 32274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:27.221071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:36071 (local address 127.12.234.62:36803)
0809 04:48:27.221575 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:27.221581 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:27.222533 (+ 952us) server_negotiation.cc:407] Connection header received
0809 04:48:27.224436 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:27.224441 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:27.224521 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:27.224645 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:27.226044 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:27.226718 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:27.228449 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:27.229009 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:27.229160 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:27.232412 (+ 3252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:27.232418 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:27.232421 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:27.232450 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:27.235125 (+ 2675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:27.235860 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:27.235868 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:27.235873 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:27.236039 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:27.236473 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:27.236481 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:27.236487 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:27.236850 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:27.237007 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:27.237583 (+ 576us) server_negotiation.cc:299] Negotiation successful
0809 04:48:27.237831 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":239,"threads_started":1}
I20260809 04:48:27.238852 32275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:27.221602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:36803 (local address 127.12.234.2:36071)
0809 04:48:27.222378 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:27.222419 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:27.224181 (+ 1762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:27.224859 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:27.224866 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:27.225329 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:48:27.225873 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:27.225902 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:27.226873 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:27.226877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:27.228303 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:27.228315 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:27.229257 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:27.229277 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:27.229489 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:27.230155 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:48:27.230176 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:48:27.232192 (+ 2016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:48:27.235277 (+ 3085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:27.235283 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:27.235293 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:27.235725 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:27.236166 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:27.236169 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:27.236171 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:27.236346 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:27.236962 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:48:27.236967 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:48:27.237221 (+ 254us) client_negotiation.cc:769] Sending connection context
0809 04:48:27.237447 (+ 226us) client_negotiation.cc:240] Negotiation successful
0809 04:48:27.237798 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":327,"threads_started":1}
I20260809 04:48:27.240314 32272 heartbeater.cc:347] Connected to a master server at 127.12.234.62:36803
I20260809 04:48:27.240634 32272 heartbeater.cc:464] Registering TS with master...
I20260809 04:48:27.241331 32272 heartbeater.cc:511] Master 127.12.234.62:36803 requested a full tablet report, sending...
I20260809 04:48:27.242770 31917 ts_manager.cc:194] Registered new tserver with Master: 8a044a6257334b8c90423d8cc75bf54b (127.12.234.2:38249)
I20260809 04:48:27.243577 31917 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.2@KRBTEST.COM'} at 127.12.234.2:36071
WARNING: no policy specified for kudu/127.12.234.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
I20260809 04:48:27.285316 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:0
--local_ip_for_outbound_sockets=127.12.234.3
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:36803
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:33583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:48:27.633104 32278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:27.633419 32278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:27.633599 32278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:27.641325 32278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:27.641463 32278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:27.641633 32278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:48:27.651485 32278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:33583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:36803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:27.652731 32278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:27.653986 32278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:27.666522 32285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:27.666579 32284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:27.666621 32287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:27.667424 32278 server_base.cc:1034] running on GCE node
I20260809 04:48:27.667925 32278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:27.668823 32278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:27.670011 32278 hybrid_clock.cc:648] HybridClock initialized: now 1786250907669937 us; error 51 us; skew 500 ppm
I20260809 04:48:27.670315 32278 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:48:27 dist-test-slave-1jjb krb5kdc[31869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250907, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:27.673362 32278 init.cc:377] Logged in from keytab as kudu/127.12.234.3@KRBTEST.COM (short username kudu)
I20260809 04:48:27.674700 32278 webserver.cc:460] Webserver started at http://127.12.234.3:32955/ using document root <none> and password file <none>
I20260809 04:48:27.675314 32278 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:27.675442 32278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:27.675787 32278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:27.678647 32278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-2/data/instance:
uuid: "d2bc72e3b8514a70a8adbb3b02911f2e"
format_stamp: "Formatted at 2026-08-09 04:48:27 on dist-test-slave-1jjb"
server_key: "0a67d7961fc99351a8b5c33d4fa3af0b"
server_key_iv: "ebc5597802fd2729572319c546388b53"
server_key_version: "encryptionkey@0"
I20260809 04:48:27.679422 32278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance:
uuid: "d2bc72e3b8514a70a8adbb3b02911f2e"
format_stamp: "Formatted at 2026-08-09 04:48:27 on dist-test-slave-1jjb"
server_key: "0a67d7961fc99351a8b5c33d4fa3af0b"
server_key_iv: "ebc5597802fd2729572319c546388b53"
server_key_version: "encryptionkey@0"
I20260809 04:48:27.684633 32278 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 04:48:27.687872 32294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:27.689318 32278 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:48:27.689471 32278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "d2bc72e3b8514a70a8adbb3b02911f2e"
format_stamp: "Formatted at 2026-08-09 04:48:27 on dist-test-slave-1jjb"
server_key: "0a67d7961fc99351a8b5c33d4fa3af0b"
server_key_iv: "ebc5597802fd2729572319c546388b53"
server_key_version: "encryptionkey@0"
I20260809 04:48:27.689651 32278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:27.701113 32278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:27.704861 32278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:27.705147 32278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:27.706032 32278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:27.707265 32278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:48:27.707337 32278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:27.707455 32278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:27.707559 32278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:27.721871 32127 heartbeater.cc:503] Master 127.12.234.62:36803 was elected leader, sending a full tablet report...
I20260809 04:48:27.740782 32278 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:38419
I20260809 04:48:27.740824 32419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:38419 every 8 connection(s)
I20260809 04:48:27.742013 32278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:48:27.751386 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 32278
I20260809 04:48:27.751564 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance
I20260809 04:48:27.751997 13224 external_mini_cluster.cc:1428] Setting key 204dfdbc35e3b97b829fe91765898521
Aug 09 04:48:27 dist-test-slave-1jjb krb5kdc[31869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250907, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:48:27.761036 32423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:27.744412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:60713 (local address 127.12.234.62:36803)
0809 04:48:27.744881 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:27.744885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:27.745705 (+ 820us) server_negotiation.cc:407] Connection header received
0809 04:48:27.747719 (+ 2014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:27.747723 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:27.747798 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:27.747892 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:27.749341 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:27.750105 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:27.751622 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:27.752067 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:27.752255 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:27.755811 (+ 3556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:27.755816 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:27.755819 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:27.755854 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:27.758484 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:27.759100 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:27.759104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:27.759106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:27.759244 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:27.759605 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:27.759608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:27.759610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:27.759851 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:27.759963 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:27.760602 (+ 639us) server_negotiation.cc:299] Negotiation successful
0809 04:48:27.760761 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":251,"threads_started":1}
I20260809 04:48:27.761898 32424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:27.744865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:36803 (local address 127.12.234.3:60713)
0809 04:48:27.745541 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:27.745583 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:27.747483 (+ 1900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:27.748137 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:27.748145 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:27.748653 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:48:27.749194 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:27.749211 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:27.750241 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:27.750258 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:27.751468 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:27.751485 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:27.752351 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:27.752359 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:27.752648 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:27.753367 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:48:27.753388 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:48:27.755641 (+ 2253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:48:27.758604 (+ 2963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:27.758610 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:27.758620 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:27.759002 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:27.759341 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:27.759344 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:27.759346 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:27.759515 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:27.759950 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:48:27.759955 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:48:27.760291 (+ 336us) client_negotiation.cc:769] Sending connection context
0809 04:48:27.760508 (+ 217us) client_negotiation.cc:240] Negotiation successful
0809 04:48:27.760835 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":252,"threads_started":1}
I20260809 04:48:27.763245 32420 heartbeater.cc:347] Connected to a master server at 127.12.234.62:36803
I20260809 04:48:27.763512 32420 heartbeater.cc:464] Registering TS with master...
I20260809 04:48:27.764148 32420 heartbeater.cc:511] Master 127.12.234.62:36803 requested a full tablet report, sending...
I20260809 04:48:27.765550 31917 ts_manager.cc:194] Registered new tserver with Master: d2bc72e3b8514a70a8adbb3b02911f2e (127.12.234.3:38419)
I20260809 04:48:27.766525 31917 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.3@KRBTEST.COM'} at 127.12.234.3:60713
I20260809 04:48:27.767089 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:48:27.783187 32423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:27.771604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44994 (local address 127.12.234.62:36803)
0809 04:48:27.771833 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:27.771841 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:27.772217 (+ 376us) server_negotiation.cc:407] Connection header received
0809 04:48:27.772340 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:27.772344 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:27.772411 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:27.772483 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:27.773878 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:27.774523 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:27.775724 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:27.776140 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:27.776227 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:27.778390 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:27.778395 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:27.778398 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:27.778424 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:27.780956 (+ 2532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:27.781565 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:27.781569 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:27.781571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:27.781671 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:27.781977 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:27.781980 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:27.781982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:27.782256 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:27.782434 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:27.782703 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 04:48:27.782850 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 04:48:27.786805 31917 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44994:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:48:27.793604 31917 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:48:27.831931 32439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:27.805872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37394 (local address 127.12.234.2:38249)
0809 04:48:27.824543 (+ 18671us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:27.824550 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:27.824573 (+ 23us) server_negotiation.cc:407] Connection header received
0809 04:48:27.824657 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:27.824662 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:27.824832 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:27.825047 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:27.825787 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:27.827144 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:27.829757 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:27.830697 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:27.830873 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:27.831071 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:27.831444 (+ 373us) server_negotiation.cc:299] Negotiation successful
0809 04:48:27.831614 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18519,"thread_start_us":18431,"threads_started":1}
I20260809 04:48:27.831981 32435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:27.805347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:38249 (local address 127.0.0.1:37394)
0809 04:48:27.821685 (+ 16338us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:27.821699 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:27.821786 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:27.825048 (+ 3262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:27.825055 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:27.825091 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:27.825584 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:27.825598 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:27.827295 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:27.827300 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:27.829597 (+ 2297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:27.829610 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:27.830952 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:27.830963 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:27.831158 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:27.831236 (+ 78us) client_negotiation.cc:769] Sending connection context
0809 04:48:27.831377 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 04:48:27.831555 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16245,"mutex_wait_us":24,"thread_start_us":20042,"threads_started":1}
I20260809 04:48:27.833550 32438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:27.804843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:36659 (local address 127.0.0.1:49582)
0809 04:48:27.817980 (+ 13137us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:27.818015 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:27.818207 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:27.825835 (+ 7628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:27.825839 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:27.825869 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:27.826297 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:27.826309 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:27.828429 (+ 2120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:27.828434 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:27.830824 (+ 2390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:27.830857 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:27.832752 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:27.832759 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:27.832948 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:27.832984 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 04:48:27.833101 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 04:48:27.833219 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12907,"thread_start_us":16617,"threads_started":1}
I20260809 04:48:27.833979 32437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:27.805550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:38419 (local address 127.0.0.1:52280)
0809 04:48:27.825709 (+ 20159us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:27.825727 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:27.825841 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:27.826447 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:27.826451 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:27.826476 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:27.826791 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:27.826801 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:27.828805 (+ 2004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:27.828810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:27.831103 (+ 2293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:27.831117 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:27.833068 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:27.833074 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:27.833224 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:27.833252 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 04:48:27.833347 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 04:48:27.833462 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20035,"mutex_wait_us":24,"thread_start_us":12060,"threads_started":1}
I20260809 04:48:27.834528 32436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:27.805247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49582 (local address 127.12.234.1:36659)
0809 04:48:27.824558 (+ 19311us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:27.824571 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:27.824607 (+ 36us) server_negotiation.cc:407] Connection header received
0809 04:48:27.824726 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:27.824736 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:27.825018 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:27.826497 (+ 1479us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:27.826716 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:27.828264 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:27.831068 (+ 2804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:27.832551 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:27.832702 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:27.832914 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:27.834144 (+ 1230us) server_negotiation.cc:299] Negotiation successful
0809 04:48:27.834299 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19134,"thread_start_us":18973,"threads_started":1}
I20260809 04:48:27.835255 32441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:27.805813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52280 (local address 127.12.234.3:38419)
0809 04:48:27.825710 (+ 19897us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:27.825718 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:27.825829 (+ 111us) server_negotiation.cc:407] Connection header received
0809 04:48:27.826035 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:27.826043 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:27.826245 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:27.826442 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:27.826974 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:27.828619 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:27.831269 (+ 2650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:27.832735 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:27.832852 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:27.833051 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:27.834793 (+ 1742us) server_negotiation.cc:299] Negotiation successful
0809 04:48:27.834945 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19727,"thread_start_us":19783,"threads_started":1}
I20260809 04:48:27.836032 32199 tablet_service.cc:1467] Processing CreateTablet for tablet 08ac885aee164414ad13171c131e1bcc (DEFAULT_TABLE table=test-table [id=869b32a8f0a24acabab0c4a7fae36e85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:48:27.837373 32199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08ac885aee164414ad13171c131e1bcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:27.837918 32056 tablet_service.cc:1467] Processing CreateTablet for tablet 08ac885aee164414ad13171c131e1bcc (DEFAULT_TABLE table=test-table [id=869b32a8f0a24acabab0c4a7fae36e85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:48:27.838611 32349 tablet_service.cc:1467] Processing CreateTablet for tablet 08ac885aee164414ad13171c131e1bcc (DEFAULT_TABLE table=test-table [id=869b32a8f0a24acabab0c4a7fae36e85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:48:27.839324 32056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08ac885aee164414ad13171c131e1bcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:27.839994 32349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08ac885aee164414ad13171c131e1bcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:27.848721 32442 tablet_bootstrap.cc:492] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b: Bootstrap starting.
I20260809 04:48:27.851744 32442 tablet_bootstrap.cc:654] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:27.852872 32442 log.cc:824] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:27.853567 32443 tablet_bootstrap.cc:492] T 08ac885aee164414ad13171c131e1bcc P 11c92879d2354be390e4362c076a267b: Bootstrap starting.
I20260809 04:48:27.855090 32444 tablet_bootstrap.cc:492] T 08ac885aee164414ad13171c131e1bcc P d2bc72e3b8514a70a8adbb3b02911f2e: Bootstrap starting.
I20260809 04:48:27.857295 32443 tablet_bootstrap.cc:654] T 08ac885aee164414ad13171c131e1bcc P 11c92879d2354be390e4362c076a267b: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:27.857913 32444 tablet_bootstrap.cc:654] T 08ac885aee164414ad13171c131e1bcc P d2bc72e3b8514a70a8adbb3b02911f2e: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:27.858431 32442 tablet_bootstrap.cc:492] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b: No bootstrap required, opened a new log
I20260809 04:48:27.858588 32443 log.cc:824] T 08ac885aee164414ad13171c131e1bcc P 11c92879d2354be390e4362c076a267b: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:27.858714 32442 ts_tablet_manager.cc:1397] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260809 04:48:27.858955 32444 log.cc:824] T 08ac885aee164414ad13171c131e1bcc P d2bc72e3b8514a70a8adbb3b02911f2e: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:27.861346 32443 tablet_bootstrap.cc:492] T 08ac885aee164414ad13171c131e1bcc P 11c92879d2354be390e4362c076a267b: No bootstrap required, opened a new log
I20260809 04:48:27.861598 32443 ts_tablet_manager.cc:1397] T 08ac885aee164414ad13171c131e1bcc P 11c92879d2354be390e4362c076a267b: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 04:48:27.861804 32444 tablet_bootstrap.cc:492] T 08ac885aee164414ad13171c131e1bcc P d2bc72e3b8514a70a8adbb3b02911f2e: No bootstrap required, opened a new log
I20260809 04:48:27.862046 32444 ts_tablet_manager.cc:1397] T 08ac885aee164414ad13171c131e1bcc P d2bc72e3b8514a70a8adbb3b02911f2e: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 04:48:27.866055 32442 raft_consensus.cc:348] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c92879d2354be390e4362c076a267b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36659 } } peers { permanent_uuid: "8a044a6257334b8c90423d8cc75bf54b" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38249 } } peers { permanent_uuid: "d2bc72e3b8514a70a8adbb3b02911f2e" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 38419 } }
I20260809 04:48:27.866401 32442 raft_consensus.cc:374] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:27.866560 32442 raft_consensus.cc:729] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a044a6257334b8c90423d8cc75bf54b, State: Initialized, Role: FOLLOWER
I20260809 04:48:27.867385 32442 consensus_queue.cc:260] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c92879d2354be390e4362c076a267b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36659 } } peers { permanent_uuid: "8a044a6257334b8c90423d8cc75bf54b" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38249 } } peers { permanent_uuid: "d2bc72e3b8514a70a8adbb3b02911f2e" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 38419 } }
I20260809 04:48:27.872262 32443 raft_consensus.cc:348] T 08ac885aee164414ad13171c131e1bcc P 11c92879d2354be390e4362c076a267b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c92879d2354be390e4362c076a267b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36659 } } peers { permanent_uuid: "8a044a6257334b8c90423d8cc75bf54b" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38249 } } peers { permanent_uuid: "d2bc72e3b8514a70a8adbb3b02911f2e" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 38419 } }
I20260809 04:48:27.872699 32443 raft_consensus.cc:374] T 08ac885aee164414ad13171c131e1bcc P 11c92879d2354be390e4362c076a267b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:27.872483 32444 raft_consensus.cc:348] T 08ac885aee164414ad13171c131e1bcc P d2bc72e3b8514a70a8adbb3b02911f2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c92879d2354be390e4362c076a267b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36659 } } peers { permanent_uuid: "8a044a6257334b8c90423d8cc75bf54b" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38249 } } peers { permanent_uuid: "d2bc72e3b8514a70a8adbb3b02911f2e" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 38419 } }
I20260809 04:48:27.872807 32443 raft_consensus.cc:729] T 08ac885aee164414ad13171c131e1bcc P 11c92879d2354be390e4362c076a267b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11c92879d2354be390e4362c076a267b, State: Initialized, Role: FOLLOWER
I20260809 04:48:27.872890 32444 raft_consensus.cc:374] T 08ac885aee164414ad13171c131e1bcc P d2bc72e3b8514a70a8adbb3b02911f2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:27.872989 32444 raft_consensus.cc:729] T 08ac885aee164414ad13171c131e1bcc P d2bc72e3b8514a70a8adbb3b02911f2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2bc72e3b8514a70a8adbb3b02911f2e, State: Initialized, Role: FOLLOWER
I20260809 04:48:27.873618 32272 heartbeater.cc:503] Master 127.12.234.62:36803 was elected leader, sending a full tablet report...
I20260809 04:48:27.873466 32443 consensus_queue.cc:260] T 08ac885aee164414ad13171c131e1bcc P 11c92879d2354be390e4362c076a267b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c92879d2354be390e4362c076a267b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36659 } } peers { permanent_uuid: "8a044a6257334b8c90423d8cc75bf54b" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38249 } } peers { permanent_uuid: "d2bc72e3b8514a70a8adbb3b02911f2e" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 38419 } }
I20260809 04:48:27.873513 32444 consensus_queue.cc:260] T 08ac885aee164414ad13171c131e1bcc P d2bc72e3b8514a70a8adbb3b02911f2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c92879d2354be390e4362c076a267b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36659 } } peers { permanent_uuid: "8a044a6257334b8c90423d8cc75bf54b" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38249 } } peers { permanent_uuid: "d2bc72e3b8514a70a8adbb3b02911f2e" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 38419 } }
I20260809 04:48:27.875394 32443 ts_tablet_manager.cc:1428] T 08ac885aee164414ad13171c131e1bcc P 11c92879d2354be390e4362c076a267b: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260809 04:48:27.876475 32420 heartbeater.cc:503] Master 127.12.234.62:36803 was elected leader, sending a full tablet report...
I20260809 04:48:27.876997 32444 ts_tablet_manager.cc:1428] T 08ac885aee164414ad13171c131e1bcc P d2bc72e3b8514a70a8adbb3b02911f2e: Time spent starting tablet: real 0.015s user 0.011s sys 0.005s
I20260809 04:48:27.878275 32442 ts_tablet_manager.cc:1428] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b: Time spent starting tablet: real 0.019s user 0.008s sys 0.009s
W20260809 04:48:27.942626 32128 tablet.cc:2369] T 08ac885aee164414ad13171c131e1bcc P 11c92879d2354be390e4362c076a267b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:48:27.970525 32273 tablet.cc:2369] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:48:27.993659 32422 tablet.cc:2369] T 08ac885aee164414ad13171c131e1bcc P d2bc72e3b8514a70a8adbb3b02911f2e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:48:28.022351 32448 raft_consensus.cc:482] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:48:28.022703 32448 raft_consensus.cc:504] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c92879d2354be390e4362c076a267b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36659 } } peers { permanent_uuid: "8a044a6257334b8c90423d8cc75bf54b" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38249 } } peers { permanent_uuid: "d2bc72e3b8514a70a8adbb3b02911f2e" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 38419 } }
I20260809 04:48:28.024438 32448 leader_election.cc:288] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11c92879d2354be390e4362c076a267b (127.12.234.1:36659), d2bc72e3b8514a70a8adbb3b02911f2e (127.12.234.3:38419)
I20260809 04:48:28.031680 32441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:28.024961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:44473 (local address 127.12.234.3:38419)
0809 04:48:28.025181 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:28.025190 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:28.026110 (+ 920us) server_negotiation.cc:407] Connection header received
0809 04:48:28.026299 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:28.026304 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:28.026384 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:28.026501 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:28.027107 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:28.027922 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:28.030033 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:28.030777 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:28.030957 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:28.031028 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:28.031320 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 04:48:28.031446 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 04:48:28.031700 32436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:28.025044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:36653 (local address 127.12.234.1:36659)
0809 04:48:28.025292 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:28.025300 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:28.025938 (+ 638us) server_negotiation.cc:407] Connection header received
0809 04:48:28.026106 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:28.026112 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:28.026192 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:28.026287 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:28.026973 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:28.027944 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:28.029973 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:28.030761 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:28.030902 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:28.030985 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:28.031353 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 04:48:28.031472 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 04:48:28.031814 32451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:28.024782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:36659 (local address 127.12.234.2:36653)
0809 04:48:28.025804 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:28.025819 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:28.025966 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:28.026316 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:28.026320 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:28.026345 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:28.026819 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:28.026833 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:28.028152 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:28.028156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:28.029813 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:28.029826 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:28.030991 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:28.030997 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:28.031178 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:28.031209 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 04:48:28.031311 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 04:48:28.031423 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":854,"mutex_wait_us":27,"thread_start_us":624,"threads_started":1}
I20260809 04:48:28.032032 32452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:28.024890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:38419 (local address 127.12.234.2:44473)
0809 04:48:28.025980 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:28.026002 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:28.026144 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:28.026530 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:28.026533 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:28.026552 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:28.026925 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:28.026937 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:28.028057 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:28.028063 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:28.029841 (+ 1778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:28.029854 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:28.031031 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:28.031036 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:28.031167 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:28.031198 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 04:48:28.031264 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 04:48:28.031390 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":953,"mutex_wait_us":55,"thread_start_us":577,"threads_started":1}
I20260809 04:48:28.032904 32079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08ac885aee164414ad13171c131e1bcc" candidate_uuid: "8a044a6257334b8c90423d8cc75bf54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11c92879d2354be390e4362c076a267b" is_pre_election: true
I20260809 04:48:28.033046 32369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08ac885aee164414ad13171c131e1bcc" candidate_uuid: "8a044a6257334b8c90423d8cc75bf54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2bc72e3b8514a70a8adbb3b02911f2e" is_pre_election: true
I20260809 04:48:28.033412 32079 raft_consensus.cc:2454] T 08ac885aee164414ad13171c131e1bcc P 11c92879d2354be390e4362c076a267b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a044a6257334b8c90423d8cc75bf54b in term 0.
I20260809 04:48:28.033550 32369 raft_consensus.cc:2454] T 08ac885aee164414ad13171c131e1bcc P d2bc72e3b8514a70a8adbb3b02911f2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a044a6257334b8c90423d8cc75bf54b in term 0.
I20260809 04:48:28.034039 32153 leader_election.cc:302] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11c92879d2354be390e4362c076a267b, 8a044a6257334b8c90423d8cc75bf54b; no voters:
I20260809 04:48:28.034431 32448 raft_consensus.cc:2781] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:48:28.034566 32448 raft_consensus.cc:482] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:48:28.034675 32448 raft_consensus.cc:3037] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:28.036720 32448 raft_consensus.cc:504] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c92879d2354be390e4362c076a267b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36659 } } peers { permanent_uuid: "8a044a6257334b8c90423d8cc75bf54b" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38249 } } peers { permanent_uuid: "d2bc72e3b8514a70a8adbb3b02911f2e" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 38419 } }
I20260809 04:48:28.037305 32448 leader_election.cc:288] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b [CANDIDATE]: Term 1 election: Requested vote from peers 11c92879d2354be390e4362c076a267b (127.12.234.1:36659), d2bc72e3b8514a70a8adbb3b02911f2e (127.12.234.3:38419)
I20260809 04:48:28.037684 32079 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08ac885aee164414ad13171c131e1bcc" candidate_uuid: "8a044a6257334b8c90423d8cc75bf54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11c92879d2354be390e4362c076a267b"
I20260809 04:48:28.037760 32369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08ac885aee164414ad13171c131e1bcc" candidate_uuid: "8a044a6257334b8c90423d8cc75bf54b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2bc72e3b8514a70a8adbb3b02911f2e"
I20260809 04:48:28.037875 32079 raft_consensus.cc:3037] T 08ac885aee164414ad13171c131e1bcc P 11c92879d2354be390e4362c076a267b [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:28.037982 32369 raft_consensus.cc:3037] T 08ac885aee164414ad13171c131e1bcc P d2bc72e3b8514a70a8adbb3b02911f2e [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:28.040524 32369 raft_consensus.cc:2454] T 08ac885aee164414ad13171c131e1bcc P d2bc72e3b8514a70a8adbb3b02911f2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a044a6257334b8c90423d8cc75bf54b in term 1.
I20260809 04:48:28.040568 32079 raft_consensus.cc:2454] T 08ac885aee164414ad13171c131e1bcc P 11c92879d2354be390e4362c076a267b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a044a6257334b8c90423d8cc75bf54b in term 1.
I20260809 04:48:28.040931 32150 leader_election.cc:302] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a044a6257334b8c90423d8cc75bf54b, d2bc72e3b8514a70a8adbb3b02911f2e; no voters:
I20260809 04:48:28.041201 32448 raft_consensus.cc:2781] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:48:28.041572 32448 raft_consensus.cc:686] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b [term 1 LEADER]: Becoming Leader. State: Replica: 8a044a6257334b8c90423d8cc75bf54b, State: Running, Role: LEADER
I20260809 04:48:28.042027 32448 consensus_queue.cc:237] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c92879d2354be390e4362c076a267b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36659 } } peers { permanent_uuid: "8a044a6257334b8c90423d8cc75bf54b" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38249 } } peers { permanent_uuid: "d2bc72e3b8514a70a8adbb3b02911f2e" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 38419 } }
I20260809 04:48:28.046075 31915 catalog_manager.cc:5495] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a044a6257334b8c90423d8cc75bf54b (127.12.234.2). New cstate: current_term: 1 leader_uuid: "8a044a6257334b8c90423d8cc75bf54b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11c92879d2354be390e4362c076a267b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36659 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a044a6257334b8c90423d8cc75bf54b" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 38249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d2bc72e3b8514a70a8adbb3b02911f2e" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 38419 } health_report { overall_health: UNKNOWN } } }
I20260809 04:48:28.107482 32439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:28.102404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37400 (local address 127.12.234.2:38249)
0809 04:48:28.102601 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:28.102606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:28.102704 (+ 98us) server_negotiation.cc:407] Connection header received
0809 04:48:28.102923 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:28.102927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:28.103005 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:28.103165 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:48:28.103532 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:28.104234 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:28.105532 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:28.105932 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:28.106051 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:28.106496 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:48:28.106930 (+ 434us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:48:28.107069 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:28.107121 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 04:48:28.107208 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 04:48:28.115178 32369 raft_consensus.cc:1261] T 08ac885aee164414ad13171c131e1bcc P d2bc72e3b8514a70a8adbb3b02911f2e [term 1 FOLLOWER]: Refusing update from remote peer 8a044a6257334b8c90423d8cc75bf54b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:48:28.115263 32079 raft_consensus.cc:1261] T 08ac885aee164414ad13171c131e1bcc P 11c92879d2354be390e4362c076a267b [term 1 FOLLOWER]: Refusing update from remote peer 8a044a6257334b8c90423d8cc75bf54b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:48:28.116127 32453 consensus_queue.cc:1035] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "11c92879d2354be390e4362c076a267b" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 36659 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:48:28.116400 32448 consensus_queue.cc:1035] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2bc72e3b8514a70a8adbb3b02911f2e" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 38419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:48:28.143664 31917 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:48:28.143980 31917 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:48:28.147569 31917 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 543156f42da845989b08172ded5c5f95: Sending DeleteTablet for 3 replicas of tablet 08ac885aee164414ad13171c131e1bcc
I20260809 04:48:28.148689 32056 tablet_service.cc:1514] Processing DeleteTablet for tablet 08ac885aee164414ad13171c131e1bcc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:48:28 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:49582
I20260809 04:48:28.148833 32199 tablet_service.cc:1514] Processing DeleteTablet for tablet 08ac885aee164414ad13171c131e1bcc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:48:28 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:37394
I20260809 04:48:28.149530 32349 tablet_service.cc:1514] Processing DeleteTablet for tablet 08ac885aee164414ad13171c131e1bcc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:48:28 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:52280
I20260809 04:48:28.150727 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 31989
I20260809 04:48:28.169173 32467 tablet_replica.cc:330] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b: stopping tablet replica
I20260809 04:48:28.169651 32467 raft_consensus.cc:2229] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:48:28.171995 32467 raft_consensus.cc:2258] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 09 04:48:28 dist-test-slave-1jjb krb5kdc[31869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.1@KRBTEST.COM
I20260809 04:48:28.177445 32467 ts_tablet_manager.cc:1905] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:48:28.180666 32468 tablet_replica.cc:330] T 08ac885aee164414ad13171c131e1bcc P d2bc72e3b8514a70a8adbb3b02911f2e: stopping tablet replica
I20260809 04:48:28.180974 32466 tablet_replica.cc:330] T 08ac885aee164414ad13171c131e1bcc P 11c92879d2354be390e4362c076a267b: stopping tablet replica
I20260809 04:48:28.181157 32468 raft_consensus.cc:2229] T 08ac885aee164414ad13171c131e1bcc P d2bc72e3b8514a70a8adbb3b02911f2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:48:28.181455 32466 raft_consensus.cc:2229] T 08ac885aee164414ad13171c131e1bcc P 11c92879d2354be390e4362c076a267b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:48:28.181757 32468 raft_consensus.cc:2258] T 08ac885aee164414ad13171c131e1bcc P d2bc72e3b8514a70a8adbb3b02911f2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:48:28.181959 32466 raft_consensus.cc:2258] T 08ac885aee164414ad13171c131e1bcc P 11c92879d2354be390e4362c076a267b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:48:28.185477 32466 ts_tablet_manager.cc:1905] T 08ac885aee164414ad13171c131e1bcc P 11c92879d2354be390e4362c076a267b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:48:28.185559 32467 ts_tablet_manager.cc:1918] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:48:28.185729 32467 log.cc:1196] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-1/wal/wals/08ac885aee164414ad13171c131e1bcc
I20260809 04:48:28.186381 32467 ts_tablet_manager.cc:1939] T 08ac885aee164414ad13171c131e1bcc P 8a044a6257334b8c90423d8cc75bf54b: Deleting consensus metadata
I20260809 04:48:28.188527 31901 catalog_manager.cc:4841] TS 8a044a6257334b8c90423d8cc75bf54b (127.12.234.2:38249): tablet 08ac885aee164414ad13171c131e1bcc (table test-table [id=869b32a8f0a24acabab0c4a7fae36e85]) successfully deleted
I20260809 04:48:28.189193 32468 ts_tablet_manager.cc:1905] T 08ac885aee164414ad13171c131e1bcc P d2bc72e3b8514a70a8adbb3b02911f2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:48:28.193037 32436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:28.153618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49596 (local address 127.12.234.1:36659)
0809 04:48:28.153822 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:28.153830 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:28.162355 (+ 8525us) server_negotiation.cc:407] Connection header received
0809 04:48:28.162757 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:28.162762 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:28.162844 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:28.162909 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:28.164815 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:28.166264 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:28.169912 (+ 3648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:28.170580 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:28.170757 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:28.183415 (+ 12658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:28.183435 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:28.183464 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:28.183523 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:28.188909 (+ 5386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:28.189977 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:28.189992 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:28.189996 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:28.190223 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:28.190780 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:28.190786 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:28.190790 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:28.191414 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:28.191756 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:28.192122 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 04:48:28.192396 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 04:48:28.193130 32466 ts_tablet_manager.cc:1918] T 08ac885aee164414ad13171c131e1bcc P 11c92879d2354be390e4362c076a267b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:48:28.193303 32466 log.cc:1196] T 08ac885aee164414ad13171c131e1bcc P 11c92879d2354be390e4362c076a267b: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-0/wal/wals/08ac885aee164414ad13171c131e1bcc
I20260809 04:48:28.194042 32466 ts_tablet_manager.cc:1939] T 08ac885aee164414ad13171c131e1bcc P 11c92879d2354be390e4362c076a267b: Deleting consensus metadata
I20260809 04:48:28.194082 32122 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:28.197322 32468 ts_tablet_manager.cc:1918] T 08ac885aee164414ad13171c131e1bcc P d2bc72e3b8514a70a8adbb3b02911f2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:48:28.197553 32468 log.cc:1196] T 08ac885aee164414ad13171c131e1bcc P d2bc72e3b8514a70a8adbb3b02911f2e: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786250603326224-13224-0/minicluster-data/ts-2/wal/wals/08ac885aee164414ad13171c131e1bcc
I20260809 04:48:28.198261 32468 ts_tablet_manager.cc:1939] T 08ac885aee164414ad13171c131e1bcc P d2bc72e3b8514a70a8adbb3b02911f2e: Deleting consensus metadata
I20260809 04:48:28.200557 31900 catalog_manager.cc:4841] TS d2bc72e3b8514a70a8adbb3b02911f2e (127.12.234.3:38419): tablet 08ac885aee164414ad13171c131e1bcc (table test-table [id=869b32a8f0a24acabab0c4a7fae36e85]) successfully deleted
I20260809 04:48:28.319788 31903 catalog_manager.cc:4841] TS 11c92879d2354be390e4362c076a267b (127.12.234.1:36659): tablet 08ac885aee164414ad13171c131e1bcc (table test-table [id=869b32a8f0a24acabab0c4a7fae36e85]) successfully deleted
I20260809 04:48:28.319914 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 31989
I20260809 04:48:28.340910 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 32133
Aug 09 04:48:28 dist-test-slave-1jjb krb5kdc[31869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.2@KRBTEST.COM
I20260809 04:48:28.358023 32439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:28.341647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37410 (local address 127.12.234.2:38249)
0809 04:48:28.341818 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:28.341824 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:28.341844 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:48:28.341971 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:28.341976 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:28.342052 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:28.342174 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:28.343315 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:28.344557 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:28.345875 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:28.346487 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:28.346591 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:28.351231 (+ 4640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:28.351246 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:28.351269 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:28.351311 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:28.355181 (+ 3870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:28.355820 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:28.355835 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:28.355839 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:28.356011 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:28.356523 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:28.356528 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:28.356531 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:28.356932 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:28.357248 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:28.357390 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 04:48:28.357645 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 04:48:28.358815 32266 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:28.466943 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 32133
I20260809 04:48:28.489557 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 32278
Aug 09 04:48:28 dist-test-slave-1jjb krb5kdc[31869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.3@KRBTEST.COM
I20260809 04:48:28.503614 32441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:28.490337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52284 (local address 127.12.234.3:38419)
0809 04:48:28.490528 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:28.490549 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:28.490569 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:48:28.490750 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:28.490754 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:28.490823 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:28.490919 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:28.492197 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:28.492895 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:28.494165 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:28.494636 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:28.494754 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:28.499025 (+ 4271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:28.499037 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:28.499055 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:28.499089 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:28.501529 (+ 2440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:28.502085 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:28.502092 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:28.502094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:28.502227 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:28.502506 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:28.502509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:28.502511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:28.502811 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:28.503014 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:28.503236 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 04:48:28.503378 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 04:48:28.504274 32415 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:28.588930 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 32278
I20260809 04:48:28.608609 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 31882
I20260809 04:48:28.609876 31953 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:28.716414 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 31882
2026-08-09T04:48:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4458 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T04:48:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:48:28Z Disabled control of system clock
I20260809 04:48:28.806304 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:42531
--webserver_interface=127.12.234.62
--webserver_port=0
--builtin_ntp_servers=127.12.234.20:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:42531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 04:48:29.130964 32479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:29.131219 32479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:29.131356 32479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:29.137976 32479 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:48:29.138093 32479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:29.138185 32479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:29.138285 32479 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:48:29.138361 32479 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:48:29.147326 32479 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:42531
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:42531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:29.148564 32479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:29.150225 32479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:29.161262 32484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:29.161327 32485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:29.161767 32479 server_base.cc:1034] running on GCE node
W20260809 04:48:29.161314 32487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:29.162706 32479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:29.163801 32479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:29.165012 32479 hybrid_clock.cc:648] HybridClock initialized: now 1786250909164958 us; error 47 us; skew 500 ppm
I20260809 04:48:29.165378 32479 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:48:29.170595 32479 webserver.cc:460] Webserver started at http://127.12.234.62:39295/ using document root <none> and password file <none>
I20260809 04:48:29.171240 32479 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:29.171372 32479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:29.171708 32479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:29.174593 32479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "60c5c4bcbc5c49c090272ba7eec48636"
format_stamp: "Formatted at 2026-08-09 04:48:29 on dist-test-slave-1jjb"
server_key: "8150068b1599aefb1ae93a112db470f7"
server_key_iv: "381e19303835004e939bd9b2bffc362e"
server_key_version: "encryptionkey@0"
I20260809 04:48:29.175375 32479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "60c5c4bcbc5c49c090272ba7eec48636"
format_stamp: "Formatted at 2026-08-09 04:48:29 on dist-test-slave-1jjb"
server_key: "8150068b1599aefb1ae93a112db470f7"
server_key_iv: "381e19303835004e939bd9b2bffc362e"
server_key_version: "encryptionkey@0"
I20260809 04:48:29.179996 32479 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260809 04:48:29.183008 32493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:29.184389 32479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:48:29.184600 32479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "60c5c4bcbc5c49c090272ba7eec48636"
format_stamp: "Formatted at 2026-08-09 04:48:29 on dist-test-slave-1jjb"
server_key: "8150068b1599aefb1ae93a112db470f7"
server_key_iv: "381e19303835004e939bd9b2bffc362e"
server_key_version: "encryptionkey@0"
I20260809 04:48:29.184819 32479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:29.212953 32479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:29.213830 32479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:29.214125 32479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:29.232642 32479 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:42531
I20260809 04:48:29.232690 32550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:42531 every 8 connection(s)
I20260809 04:48:29.233886 32479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:48:29.237288 32551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:29.240226 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 32479
I20260809 04:48:29.240386 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:48:29.240872 13224 external_mini_cluster.cc:1428] Setting key ab7a2ca13fb384d130c3103b079e5add
I20260809 04:48:29.249008 32551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60c5c4bcbc5c49c090272ba7eec48636: Bootstrap starting.
I20260809 04:48:29.252396 32551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60c5c4bcbc5c49c090272ba7eec48636: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:29.253615 32551 log.cc:824] T 00000000000000000000000000000000 P 60c5c4bcbc5c49c090272ba7eec48636: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:29.253723 32554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:29.242788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47924 (local address 127.12.234.62:42531)
0809 04:48:29.243654 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:29.243681 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:29.243717 (+ 36us) server_negotiation.cc:407] Connection header received
0809 04:48:29.245375 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:29.245395 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:29.245800 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:29.246162 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:29.246732 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.248021 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.249559 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.250214 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.250448 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:29.251493 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:29.251500 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:29.251510 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:29.252018 (+ 508us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:29.252162 (+ 144us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:29.252184 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:29.252326 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:29.252557 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 04:48:29.252851 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":325,"threads_started":1}
I20260809 04:48:29.256888 32551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60c5c4bcbc5c49c090272ba7eec48636: No bootstrap required, opened a new log
I20260809 04:48:29.264493 32551 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60c5c4bcbc5c49c090272ba7eec48636 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c5c4bcbc5c49c090272ba7eec48636" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42531 } }
I20260809 04:48:29.264890 32551 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60c5c4bcbc5c49c090272ba7eec48636 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:29.265002 32551 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60c5c4bcbc5c49c090272ba7eec48636 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60c5c4bcbc5c49c090272ba7eec48636, State: Initialized, Role: FOLLOWER
I20260809 04:48:29.265555 32551 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60c5c4bcbc5c49c090272ba7eec48636 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c5c4bcbc5c49c090272ba7eec48636" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42531 } }
I20260809 04:48:29.265821 32551 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60c5c4bcbc5c49c090272ba7eec48636 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:48:29.265974 32551 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60c5c4bcbc5c49c090272ba7eec48636 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:48:29.266167 32551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60c5c4bcbc5c49c090272ba7eec48636 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:29.268739 32551 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60c5c4bcbc5c49c090272ba7eec48636 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c5c4bcbc5c49c090272ba7eec48636" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42531 } }
I20260809 04:48:29.269282 32551 leader_election.cc:302] T 00000000000000000000000000000000 P 60c5c4bcbc5c49c090272ba7eec48636 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60c5c4bcbc5c49c090272ba7eec48636; no voters:
I20260809 04:48:29.269939 32551 leader_election.cc:288] T 00000000000000000000000000000000 P 60c5c4bcbc5c49c090272ba7eec48636 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:48:29.270088 32556 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60c5c4bcbc5c49c090272ba7eec48636 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:48:29.270816 32556 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60c5c4bcbc5c49c090272ba7eec48636 [term 1 LEADER]: Becoming Leader. State: Replica: 60c5c4bcbc5c49c090272ba7eec48636, State: Running, Role: LEADER
I20260809 04:48:29.271399 32556 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60c5c4bcbc5c49c090272ba7eec48636 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c5c4bcbc5c49c090272ba7eec48636" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42531 } }
I20260809 04:48:29.272192 32551 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60c5c4bcbc5c49c090272ba7eec48636 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:48:29.274130 32558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c5c4bcbc5c49c090272ba7eec48636 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60c5c4bcbc5c49c090272ba7eec48636" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c5c4bcbc5c49c090272ba7eec48636" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42531 } } }
I20260809 04:48:29.274519 32558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c5c4bcbc5c49c090272ba7eec48636 [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:29.274899 32559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c5c4bcbc5c49c090272ba7eec48636 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60c5c4bcbc5c49c090272ba7eec48636. Latest consensus state: current_term: 1 leader_uuid: "60c5c4bcbc5c49c090272ba7eec48636" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c5c4bcbc5c49c090272ba7eec48636" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 42531 } } }
I20260809 04:48:29.276158 32559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c5c4bcbc5c49c090272ba7eec48636 [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:29.277390 32563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:48:29.283093 32563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:48:29.291139 32563 catalog_manager.cc:1347] Generated new cluster ID: 4ce623a022664d1dbaf8fcf1cce3bc44
I20260809 04:48:29.291303 32563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:48:29.303236 32563 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:48:29.304628 32563 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:48:29.315913 32563 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60c5c4bcbc5c49c090272ba7eec48636: Generated new TSK 0
I20260809 04:48:29.316720 32563 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:48:29.323859 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:0
--local_ip_for_outbound_sockets=127.12.234.1
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:42531
--builtin_ntp_servers=127.12.234.20:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 04:48:29.667810 32580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:29.668179 32580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:29.668336 32580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:29.676064 32580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:29.676213 32580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:29.676416 32580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:48:29.685760 32580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:42531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:29.687063 32580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:29.688977 32580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:29.701788 32589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:29.701822 32587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:29.701996 32586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:29.702883 32580 server_base.cc:1034] running on GCE node
I20260809 04:48:29.703572 32580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:29.704596 32580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:29.705811 32580 hybrid_clock.cc:648] HybridClock initialized: now 1786250909705770 us; error 26 us; skew 500 ppm
I20260809 04:48:29.706141 32580 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:48:29.708372 32580 webserver.cc:460] Webserver started at http://127.12.234.1:35411/ using document root <none> and password file <none>
I20260809 04:48:29.709080 32580 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:29.709203 32580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:29.709499 32580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:29.712338 32580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-0/data/instance:
uuid: "0f842d4a2ede4119b0e11288dc15f446"
format_stamp: "Formatted at 2026-08-09 04:48:29 on dist-test-slave-1jjb"
server_key: "8f9c0ded70796df5c38f41c0c850f563"
server_key_iv: "9ebd67a0cd814e2b2c038bf2377b3256"
server_key_version: "encryptionkey@0"
I20260809 04:48:29.713122 32580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance:
uuid: "0f842d4a2ede4119b0e11288dc15f446"
format_stamp: "Formatted at 2026-08-09 04:48:29 on dist-test-slave-1jjb"
server_key: "8f9c0ded70796df5c38f41c0c850f563"
server_key_iv: "9ebd67a0cd814e2b2c038bf2377b3256"
server_key_version: "encryptionkey@0"
I20260809 04:48:29.718039 32580 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 04:48:29.721254 32595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:29.722501 32580 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:48:29.722666 32580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "0f842d4a2ede4119b0e11288dc15f446"
format_stamp: "Formatted at 2026-08-09 04:48:29 on dist-test-slave-1jjb"
server_key: "8f9c0ded70796df5c38f41c0c850f563"
server_key_iv: "9ebd67a0cd814e2b2c038bf2377b3256"
server_key_version: "encryptionkey@0"
I20260809 04:48:29.722869 32580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:29.736784 32580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:29.737555 32580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:29.737883 32580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:29.738879 32580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:29.740447 32580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:48:29.740573 32580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:29.740698 32580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:29.740783 32580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:29.774708 32580 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:41357
I20260809 04:48:29.774804 32714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:41357 every 8 connection(s)
I20260809 04:48:29.776152 32580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:48:29.783344 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 32580
I20260809 04:48:29.783535 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance
I20260809 04:48:29.783981 13224 external_mini_cluster.cc:1428] Setting key a5b627c75a5347dfe9a56beae27adf49
I20260809 04:48:29.787145 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:0
--local_ip_for_outbound_sockets=127.12.234.2
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:42531
--builtin_ntp_servers=127.12.234.20:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 04:48:29.789677 32717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:29.778834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:45615 (local address 127.12.234.62:42531)
0809 04:48:29.779279 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:29.779283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:29.780354 (+ 1071us) server_negotiation.cc:407] Connection header received
0809 04:48:29.782081 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:29.782087 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:29.782177 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:29.782294 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:29.783344 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.784372 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.786602 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.786966 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.787082 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:29.788789 (+ 1707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:29.788793 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:29.788796 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:29.788814 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:29.788858 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:29.788869 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:29.788950 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:29.789352 (+ 402us) server_negotiation.cc:299] Negotiation successful
0809 04:48:29.789443 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":233,"threads_started":1}
I20260809 04:48:29.791014 32718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:29.779327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:42531 (local address 127.12.234.1:45615)
0809 04:48:29.780198 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:29.780239 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:29.781818 (+ 1579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:29.782530 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:29.782538 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:29.782614 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:48:29.783167 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:29.783185 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.784584 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.784589 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:29.786466 (+ 1877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:29.786482 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:29.787186 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:29.787195 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:29.787524 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:29.788450 (+ 926us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:48:29.788456 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:48:29.788511 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:48:29.788615 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:48:29.788984 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:48:29.788997 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:48:29.789008 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 04:48:29.789260 (+ 252us) client_negotiation.cc:240] Negotiation successful
0809 04:48:29.789663 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":245,"threads_started":1}
I20260809 04:48:29.792577 32715 heartbeater.cc:347] Connected to a master server at 127.12.234.62:42531
I20260809 04:48:29.792881 32715 heartbeater.cc:464] Registering TS with master...
I20260809 04:48:29.793633 32715 heartbeater.cc:511] Master 127.12.234.62:42531 requested a full tablet report, sending...
I20260809 04:48:29.795634 32513 ts_manager.cc:194] Registered new tserver with Master: 0f842d4a2ede4119b0e11288dc15f446 (127.12.234.1:41357)
I20260809 04:48:29.797256 32513 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.234.1:45615
W20260809 04:48:30.124015 32719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:30.124298 32719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:30.124375 32719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:30.131103 32719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:30.131192 32719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:30.131284 32719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:48:30.140559 32719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:42531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:30.141722 32719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:30.143386 32719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:30.156279 32726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:30.156270 32729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:30.156278 32727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:30.157068 32719 server_base.cc:1034] running on GCE node
I20260809 04:48:30.157795 32719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:30.158691 32719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:30.159924 32719 hybrid_clock.cc:648] HybridClock initialized: now 1786250910159890 us; error 55 us; skew 500 ppm
I20260809 04:48:30.160302 32719 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:48:30.162556 32719 webserver.cc:460] Webserver started at http://127.12.234.2:44845/ using document root <none> and password file <none>
I20260809 04:48:30.163223 32719 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:30.163355 32719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:30.163662 32719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:30.167014 32719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-1/data/instance:
uuid: "22b8c9a67ba14225a4d46a6f61c4fd34"
format_stamp: "Formatted at 2026-08-09 04:48:30 on dist-test-slave-1jjb"
server_key: "16fc8853b8c90ef421724dee643e84d6"
server_key_iv: "865e1b72cbb5e411efb575e3b54c0187"
server_key_version: "encryptionkey@0"
I20260809 04:48:30.167779 32719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance:
uuid: "22b8c9a67ba14225a4d46a6f61c4fd34"
format_stamp: "Formatted at 2026-08-09 04:48:30 on dist-test-slave-1jjb"
server_key: "16fc8853b8c90ef421724dee643e84d6"
server_key_iv: "865e1b72cbb5e411efb575e3b54c0187"
server_key_version: "encryptionkey@0"
I20260809 04:48:30.172570 32719 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:48:30.175887 32736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:30.177076 32719 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260809 04:48:30.177238 32719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "22b8c9a67ba14225a4d46a6f61c4fd34"
format_stamp: "Formatted at 2026-08-09 04:48:30 on dist-test-slave-1jjb"
server_key: "16fc8853b8c90ef421724dee643e84d6"
server_key_iv: "865e1b72cbb5e411efb575e3b54c0187"
server_key_version: "encryptionkey@0"
I20260809 04:48:30.177451 32719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:30.191408 32719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:30.192221 32719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:30.192535 32719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:30.193423 32719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:30.194798 32719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:48:30.194918 32719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:30.195050 32719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:30.195161 32719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:30.228289 32719 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:45687
I20260809 04:48:30.228369 393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:45687 every 8 connection(s)
I20260809 04:48:30.229633 32719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:48:30.239846 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 32719
I20260809 04:48:30.240033 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance
I20260809 04:48:30.240646 13224 external_mini_cluster.cc:1428] Setting key 3cd6a27992e324de0b5867c44e14aefc
I20260809 04:48:30.243220 32717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:30.232060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:32857 (local address 127.12.234.62:42531)
0809 04:48:30.232266 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:30.232272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:30.233764 (+ 1492us) server_negotiation.cc:407] Connection header received
0809 04:48:30.235988 (+ 2224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:30.235993 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:30.236054 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:30.236191 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:30.237365 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.238103 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.240196 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.240879 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.241009 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:30.242414 (+ 1405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:30.242420 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:30.242427 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:30.242457 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:30.242489 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:30.242503 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:30.242605 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:30.242840 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 04:48:30.242928 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 04:48:30.243692 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:0
--local_ip_for_outbound_sockets=127.12.234.3
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:42531
--builtin_ntp_servers=127.12.234.20:35335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 04:48:30.244242 396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:30.232584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:42531 (local address 127.12.234.2:32857)
0809 04:48:30.233590 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:30.233648 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:30.235715 (+ 2067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:30.236414 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:30.236423 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:30.236518 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:48:30.237203 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:30.237226 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.238260 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.238268 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:30.240055 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:30.240068 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.241083 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.241090 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:30.241345 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:30.242117 (+ 772us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:48:30.242120 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:48:30.242166 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:48:30.242251 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:48:30.242611 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:48:30.242624 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:48:30.242632 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 04:48:30.242825 (+ 193us) client_negotiation.cc:240] Negotiation successful
0809 04:48:30.243179 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":300,"threads_started":1}
I20260809 04:48:30.245604 394 heartbeater.cc:347] Connected to a master server at 127.12.234.62:42531
I20260809 04:48:30.245891 394 heartbeater.cc:464] Registering TS with master...
I20260809 04:48:30.246562 394 heartbeater.cc:511] Master 127.12.234.62:42531 requested a full tablet report, sending...
I20260809 04:48:30.247884 32513 ts_manager.cc:194] Registered new tserver with Master: 22b8c9a67ba14225a4d46a6f61c4fd34 (127.12.234.2:45687)
I20260809 04:48:30.248708 32513 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.234.2:32857
W20260809 04:48:30.575038 397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:30.575331 397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:30.575486 397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:30.582477 397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:30.582557 397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:30.582729 397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:48:30.591310 397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:35335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:42531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:30.592536 397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:30.594135 397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:30.606725 403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:30.606763 402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:30.606793 405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:30.607429 397 server_base.cc:1034] running on GCE node
I20260809 04:48:30.608119 397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:30.609031 397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:30.610301 397 hybrid_clock.cc:648] HybridClock initialized: now 1786250910610259 us; error 53 us; skew 500 ppm
I20260809 04:48:30.610612 397 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:48:30.612865 397 webserver.cc:460] Webserver started at http://127.12.234.3:39397/ using document root <none> and password file <none>
I20260809 04:48:30.613530 397 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:30.613659 397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:30.613961 397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:30.616746 397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-2/data/instance:
uuid: "efe5abfec33149a292aa7506ed2a0482"
format_stamp: "Formatted at 2026-08-09 04:48:30 on dist-test-slave-1jjb"
server_key: "8b1ca527fc0967e6ac4e15b9c556049f"
server_key_iv: "6ecf607e82fc2d62d5cc81647c25dde5"
server_key_version: "encryptionkey@0"
I20260809 04:48:30.617494 397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance:
uuid: "efe5abfec33149a292aa7506ed2a0482"
format_stamp: "Formatted at 2026-08-09 04:48:30 on dist-test-slave-1jjb"
server_key: "8b1ca527fc0967e6ac4e15b9c556049f"
server_key_iv: "6ecf607e82fc2d62d5cc81647c25dde5"
server_key_version: "encryptionkey@0"
I20260809 04:48:30.622278 397 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:48:30.625531 411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:30.626835 397 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 04:48:30.627046 397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "efe5abfec33149a292aa7506ed2a0482"
format_stamp: "Formatted at 2026-08-09 04:48:30 on dist-test-slave-1jjb"
server_key: "8b1ca527fc0967e6ac4e15b9c556049f"
server_key_iv: "6ecf607e82fc2d62d5cc81647c25dde5"
server_key_version: "encryptionkey@0"
I20260809 04:48:30.627257 397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:30.659101 397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:30.659942 397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:30.660252 397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:30.661197 397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:30.662593 397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:48:30.662717 397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:30.662848 397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:30.662935 397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:30.696305 397 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:40071
I20260809 04:48:30.696430 532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:40071 every 8 connection(s)
I20260809 04:48:30.697609 397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:48:30.704787 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 397
I20260809 04:48:30.704984 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance
I20260809 04:48:30.705436 13224 external_mini_cluster.cc:1428] Setting key a1368f0dd6234dcc86643f93ef7c2eb5
I20260809 04:48:30.710426 32717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:30.700269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:53631 (local address 127.12.234.62:42531)
0809 04:48:30.700506 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:30.700513 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:30.701921 (+ 1408us) server_negotiation.cc:407] Connection header received
0809 04:48:30.703591 (+ 1670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:30.703595 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:30.703659 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:30.703744 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:30.705064 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.705876 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.707909 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.708446 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.708539 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:30.709765 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:30.709770 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:30.709773 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:30.709790 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:30.709821 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:30.709832 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:30.709898 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:30.710063 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 04:48:30.710193 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 04:48:30.711244 535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:30.700836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:42531 (local address 127.12.234.3:53631)
0809 04:48:30.701769 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:30.701802 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:30.703348 (+ 1546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:30.703934 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:30.703940 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:30.704013 (+ 73us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:48:30.704873 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:30.704899 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.706030 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.706036 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:30.707768 (+ 1732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:30.707791 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.708642 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.708647 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:30.708832 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:30.709488 (+ 656us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:48:30.709491 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:48:30.709526 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:48:30.709592 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:48:30.709899 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:48:30.709905 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:48:30.709909 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 04:48:30.710031 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 04:48:30.710281 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":350,"threads_started":1}
I20260809 04:48:30.712558 533 heartbeater.cc:347] Connected to a master server at 127.12.234.62:42531
I20260809 04:48:30.712797 533 heartbeater.cc:464] Registering TS with master...
I20260809 04:48:30.713411 533 heartbeater.cc:511] Master 127.12.234.62:42531 requested a full tablet report, sending...
I20260809 04:48:30.714617 32513 ts_manager.cc:194] Registered new tserver with Master: efe5abfec33149a292aa7506ed2a0482 (127.12.234.3:40071)
I20260809 04:48:30.715323 32513 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.234.3:53631
I20260809 04:48:30.720183 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:48:30.729826 32717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:30.723953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47926 (local address 127.12.234.62:42531)
0809 04:48:30.724146 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:30.724150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:30.724592 (+ 442us) server_negotiation.cc:407] Connection header received
0809 04:48:30.724772 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:30.724777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:30.724851 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:30.724952 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:30.725454 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.726200 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.727569 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.727967 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.728133 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:30.729129 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:30.729133 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:30.729137 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:30.729152 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:30.729188 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:30.729199 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:30.729312 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:30.729467 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 04:48:30.729554 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:48:30.733584 32513 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47926:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:48:30.740238 32513 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:48:30.763514 547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:30.751409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:41357 (local address 127.0.0.1:38692)
0809 04:48:30.752228 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:30.752266 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:30.752491 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:30.753513 (+ 1022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:30.753519 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:30.753551 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:30.754023 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:30.754040 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.756256 (+ 2216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.756264 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:30.759586 (+ 3322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:30.759611 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.762357 (+ 2746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.762365 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:30.762649 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:30.762718 (+ 69us) client_negotiation.cc:769] Sending connection context
0809 04:48:30.762861 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 04:48:30.763095 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":796,"threads_started":1}
I20260809 04:48:30.764607 550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:30.751581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38692 (local address 127.12.234.1:41357)
0809 04:48:30.752167 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:30.752175 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:30.752399 (+ 224us) server_negotiation.cc:407] Connection header received
0809 04:48:30.752756 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:30.752764 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:30.753008 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:30.753232 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:30.754266 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.756041 (+ 1775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.759854 (+ 3813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.761762 (+ 1908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.761914 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:30.762221 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:30.763943 (+ 1722us) server_negotiation.cc:299] Negotiation successful
0809 04:48:30.764177 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":322,"threads_started":1}
I20260809 04:48:30.770953 32646 tablet_service.cc:1467] Processing CreateTablet for tablet 072854d1b25d44c2a4dbcdaefa1c7579 (DEFAULT_TABLE table=test-table [id=b27c5b3bbded4637aa51008d1eb70fa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:48:30.772629 32646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 072854d1b25d44c2a4dbcdaefa1c7579. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:30.774354 547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:30.752864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:40071 (local address 127.0.0.1:52670)
0809 04:48:30.765105 (+ 12241us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:30.765129 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:30.765247 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:30.766924 (+ 1677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:30.766931 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:30.766962 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:30.767857 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:30.767870 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.769787 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.769792 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:30.771599 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:30.771615 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.773565 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.773570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:30.773760 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:30.773793 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 04:48:30.773890 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 04:48:30.774012 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12085,"mutex_wait_us":47,"thread_start_us":12799,"threads_started":1}
I20260809 04:48:30.774730 553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:30.753202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52670 (local address 127.12.234.3:40071)
0809 04:48:30.766423 (+ 13221us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:30.766440 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:30.766467 (+ 27us) server_negotiation.cc:407] Connection header received
0809 04:48:30.766545 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:30.766551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:30.766720 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:30.767466 (+ 746us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:30.768060 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.769621 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.771744 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.773034 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.773167 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:30.773478 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:30.774173 (+ 695us) server_negotiation.cc:299] Negotiation successful
0809 04:48:30.774356 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13060,"thread_start_us":12840,"threads_started":1}
I20260809 04:48:30.775324 554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:30.753519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58362 (local address 127.12.234.2:45687)
0809 04:48:30.765901 (+ 12382us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:30.765912 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:30.766293 (+ 381us) server_negotiation.cc:407] Connection header received
0809 04:48:30.766418 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:30.766426 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:30.766655 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:30.766878 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:30.767566 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.768953 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.771261 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.772801 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.772934 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:30.773159 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:30.773803 (+ 644us) server_negotiation.cc:299] Negotiation successful
0809 04:48:30.774948 (+ 1145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12219,"thread_start_us":12103,"threads_started":1}
I20260809 04:48:30.775913 551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:30.752991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:45687 (local address 127.0.0.1:58362)
0809 04:48:30.765981 (+ 12990us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:30.766003 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:30.766149 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:30.766866 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:30.766873 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:30.766905 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:30.767357 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:30.767370 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.769095 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.769102 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:30.771062 (+ 1960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:30.771076 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.773022 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.773029 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:30.773285 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:30.773340 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 04:48:30.773442 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 04:48:30.773574 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12821,"mutex_wait_us":56,"thread_start_us":14209,"threads_started":1}
I20260809 04:48:30.778772 461 tablet_service.cc:1467] Processing CreateTablet for tablet 072854d1b25d44c2a4dbcdaefa1c7579 (DEFAULT_TABLE table=test-table [id=b27c5b3bbded4637aa51008d1eb70fa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:48:30.779222 322 tablet_service.cc:1467] Processing CreateTablet for tablet 072854d1b25d44c2a4dbcdaefa1c7579 (DEFAULT_TABLE table=test-table [id=b27c5b3bbded4637aa51008d1eb70fa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:48:30.780627 322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 072854d1b25d44c2a4dbcdaefa1c7579. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:30.781541 461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 072854d1b25d44c2a4dbcdaefa1c7579. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:30.790045 555 tablet_bootstrap.cc:492] T 072854d1b25d44c2a4dbcdaefa1c7579 P 0f842d4a2ede4119b0e11288dc15f446: Bootstrap starting.
I20260809 04:48:30.793583 555 tablet_bootstrap.cc:654] T 072854d1b25d44c2a4dbcdaefa1c7579 P 0f842d4a2ede4119b0e11288dc15f446: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:30.794138 556 tablet_bootstrap.cc:492] T 072854d1b25d44c2a4dbcdaefa1c7579 P efe5abfec33149a292aa7506ed2a0482: Bootstrap starting.
I20260809 04:48:30.795079 555 log.cc:824] T 072854d1b25d44c2a4dbcdaefa1c7579 P 0f842d4a2ede4119b0e11288dc15f446: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:30.795496 557 tablet_bootstrap.cc:492] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34: Bootstrap starting.
I20260809 04:48:30.797770 556 tablet_bootstrap.cc:654] T 072854d1b25d44c2a4dbcdaefa1c7579 P efe5abfec33149a292aa7506ed2a0482: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:30.798437 555 tablet_bootstrap.cc:492] T 072854d1b25d44c2a4dbcdaefa1c7579 P 0f842d4a2ede4119b0e11288dc15f446: No bootstrap required, opened a new log
I20260809 04:48:30.798698 555 ts_tablet_manager.cc:1397] T 072854d1b25d44c2a4dbcdaefa1c7579 P 0f842d4a2ede4119b0e11288dc15f446: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260809 04:48:30.798851 557 tablet_bootstrap.cc:654] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:30.798996 556 log.cc:824] T 072854d1b25d44c2a4dbcdaefa1c7579 P efe5abfec33149a292aa7506ed2a0482: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:30.800022 32715 heartbeater.cc:503] Master 127.12.234.62:42531 was elected leader, sending a full tablet report...
I20260809 04:48:30.800124 557 log.cc:824] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:30.808856 556 tablet_bootstrap.cc:492] T 072854d1b25d44c2a4dbcdaefa1c7579 P efe5abfec33149a292aa7506ed2a0482: No bootstrap required, opened a new log
I20260809 04:48:30.808678 555 raft_consensus.cc:348] T 072854d1b25d44c2a4dbcdaefa1c7579 P 0f842d4a2ede4119b0e11288dc15f446 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f842d4a2ede4119b0e11288dc15f446" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 41357 } } peers { permanent_uuid: "22b8c9a67ba14225a4d46a6f61c4fd34" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45687 } } peers { permanent_uuid: "efe5abfec33149a292aa7506ed2a0482" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40071 } }
I20260809 04:48:30.809170 555 raft_consensus.cc:374] T 072854d1b25d44c2a4dbcdaefa1c7579 P 0f842d4a2ede4119b0e11288dc15f446 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:30.809187 556 ts_tablet_manager.cc:1397] T 072854d1b25d44c2a4dbcdaefa1c7579 P efe5abfec33149a292aa7506ed2a0482: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.009s
I20260809 04:48:30.809296 555 raft_consensus.cc:729] T 072854d1b25d44c2a4dbcdaefa1c7579 P 0f842d4a2ede4119b0e11288dc15f446 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f842d4a2ede4119b0e11288dc15f446, State: Initialized, Role: FOLLOWER
I20260809 04:48:30.809372 557 tablet_bootstrap.cc:492] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34: No bootstrap required, opened a new log
I20260809 04:48:30.809659 557 ts_tablet_manager.cc:1397] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.004s
I20260809 04:48:30.809949 555 consensus_queue.cc:260] T 072854d1b25d44c2a4dbcdaefa1c7579 P 0f842d4a2ede4119b0e11288dc15f446 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f842d4a2ede4119b0e11288dc15f446" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 41357 } } peers { permanent_uuid: "22b8c9a67ba14225a4d46a6f61c4fd34" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45687 } } peers { permanent_uuid: "efe5abfec33149a292aa7506ed2a0482" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40071 } }
I20260809 04:48:30.811708 555 ts_tablet_manager.cc:1428] T 072854d1b25d44c2a4dbcdaefa1c7579 P 0f842d4a2ede4119b0e11288dc15f446: Time spent starting tablet: real 0.013s user 0.011s sys 0.001s
I20260809 04:48:30.817804 556 raft_consensus.cc:348] T 072854d1b25d44c2a4dbcdaefa1c7579 P efe5abfec33149a292aa7506ed2a0482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f842d4a2ede4119b0e11288dc15f446" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 41357 } } peers { permanent_uuid: "22b8c9a67ba14225a4d46a6f61c4fd34" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45687 } } peers { permanent_uuid: "efe5abfec33149a292aa7506ed2a0482" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40071 } }
I20260809 04:48:30.818172 556 raft_consensus.cc:374] T 072854d1b25d44c2a4dbcdaefa1c7579 P efe5abfec33149a292aa7506ed2a0482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:30.818276 556 raft_consensus.cc:729] T 072854d1b25d44c2a4dbcdaefa1c7579 P efe5abfec33149a292aa7506ed2a0482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe5abfec33149a292aa7506ed2a0482, State: Initialized, Role: FOLLOWER
I20260809 04:48:30.818315 557 raft_consensus.cc:348] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f842d4a2ede4119b0e11288dc15f446" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 41357 } } peers { permanent_uuid: "22b8c9a67ba14225a4d46a6f61c4fd34" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45687 } } peers { permanent_uuid: "efe5abfec33149a292aa7506ed2a0482" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40071 } }
I20260809 04:48:30.818717 557 raft_consensus.cc:374] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:30.818833 557 raft_consensus.cc:729] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22b8c9a67ba14225a4d46a6f61c4fd34, State: Initialized, Role: FOLLOWER
I20260809 04:48:30.818885 556 consensus_queue.cc:260] T 072854d1b25d44c2a4dbcdaefa1c7579 P efe5abfec33149a292aa7506ed2a0482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f842d4a2ede4119b0e11288dc15f446" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 41357 } } peers { permanent_uuid: "22b8c9a67ba14225a4d46a6f61c4fd34" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45687 } } peers { permanent_uuid: "efe5abfec33149a292aa7506ed2a0482" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40071 } }
I20260809 04:48:30.819380 557 consensus_queue.cc:260] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f842d4a2ede4119b0e11288dc15f446" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 41357 } } peers { permanent_uuid: "22b8c9a67ba14225a4d46a6f61c4fd34" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45687 } } peers { permanent_uuid: "efe5abfec33149a292aa7506ed2a0482" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40071 } }
I20260809 04:48:30.820035 533 heartbeater.cc:503] Master 127.12.234.62:42531 was elected leader, sending a full tablet report...
I20260809 04:48:30.820552 394 heartbeater.cc:503] Master 127.12.234.62:42531 was elected leader, sending a full tablet report...
I20260809 04:48:30.820734 556 ts_tablet_manager.cc:1428] T 072854d1b25d44c2a4dbcdaefa1c7579 P efe5abfec33149a292aa7506ed2a0482: Time spent starting tablet: real 0.011s user 0.007s sys 0.005s
I20260809 04:48:30.821167 557 ts_tablet_manager.cc:1428] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34: Time spent starting tablet: real 0.011s user 0.005s sys 0.007s
I20260809 04:48:30.861482 564 raft_consensus.cc:482] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:48:30.861886 564 raft_consensus.cc:504] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f842d4a2ede4119b0e11288dc15f446" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 41357 } } peers { permanent_uuid: "22b8c9a67ba14225a4d46a6f61c4fd34" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45687 } } peers { permanent_uuid: "efe5abfec33149a292aa7506ed2a0482" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40071 } }
I20260809 04:48:30.863538 564 leader_election.cc:288] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f842d4a2ede4119b0e11288dc15f446 (127.12.234.1:41357), efe5abfec33149a292aa7506ed2a0482 (127.12.234.3:40071)
I20260809 04:48:30.870379 553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:30.864061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:43889 (local address 127.12.234.3:40071)
0809 04:48:30.864278 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:30.864287 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:30.865128 (+ 841us) server_negotiation.cc:407] Connection header received
0809 04:48:30.865310 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:30.865315 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:30.865372 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:30.865469 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:30.866048 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.866753 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.868679 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.869474 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.869672 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:30.869744 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:30.870008 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 04:48:30.870127 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:48:30.870435 550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:30.863986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:48425 (local address 127.12.234.1:41357)
0809 04:48:30.864231 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:30.864239 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:30.865057 (+ 818us) server_negotiation.cc:407] Connection header received
0809 04:48:30.865286 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:30.865290 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:30.865359 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:30.865456 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:30.866146 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.866837 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.868739 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.869523 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.869691 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:30.869756 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:30.870041 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 04:48:30.870164 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 04:48:30.870548 566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:30.863977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:40071 (local address 127.12.234.2:43889)
0809 04:48:30.865037 (+ 1060us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:30.865051 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:30.865156 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:30.865486 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:30.865490 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:30.865512 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:30.865886 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:30.865900 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.866875 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.866883 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:30.868488 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:30.868500 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.869672 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.869678 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:30.869845 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:30.869868 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 04:48:30.869971 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 04:48:30.870099 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":947,"mutex_wait_us":3,"spinlock_wait_cycles":24832,"thread_start_us":576,"threads_started":1}
I20260809 04:48:30.870596 565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:30.863964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:41357 (local address 127.12.234.2:48425)
0809 04:48:30.864945 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:30.864961 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:30.865131 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:30.865489 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:30.865493 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:30.865581 (+ 88us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:30.866003 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:30.866012 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.866984 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.866990 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:30.868534 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:30.868547 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.869715 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.869721 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:30.869884 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:30.869906 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 04:48:30.870011 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 04:48:30.870127 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":819,"mutex_wait_us":1,"spinlock_wait_cycles":20352,"thread_start_us":509,"threads_started":1}
I20260809 04:48:30.871680 485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "072854d1b25d44c2a4dbcdaefa1c7579" candidate_uuid: "22b8c9a67ba14225a4d46a6f61c4fd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe5abfec33149a292aa7506ed2a0482" is_pre_election: true
I20260809 04:48:30.871841 32667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "072854d1b25d44c2a4dbcdaefa1c7579" candidate_uuid: "22b8c9a67ba14225a4d46a6f61c4fd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f842d4a2ede4119b0e11288dc15f446" is_pre_election: true
I20260809 04:48:30.872395 32667 raft_consensus.cc:2454] T 072854d1b25d44c2a4dbcdaefa1c7579 P 0f842d4a2ede4119b0e11288dc15f446 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22b8c9a67ba14225a4d46a6f61c4fd34 in term 0.
I20260809 04:48:30.872401 485 raft_consensus.cc:2454] T 072854d1b25d44c2a4dbcdaefa1c7579 P efe5abfec33149a292aa7506ed2a0482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22b8c9a67ba14225a4d46a6f61c4fd34 in term 0.
I20260809 04:48:30.873111 32737 leader_election.cc:302] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0f842d4a2ede4119b0e11288dc15f446, 22b8c9a67ba14225a4d46a6f61c4fd34, efe5abfec33149a292aa7506ed2a0482; no voters:
I20260809 04:48:30.873558 564 raft_consensus.cc:2781] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:48:30.873720 564 raft_consensus.cc:482] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:48:30.873837 564 raft_consensus.cc:3037] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:30.876056 564 raft_consensus.cc:504] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f842d4a2ede4119b0e11288dc15f446" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 41357 } } peers { permanent_uuid: "22b8c9a67ba14225a4d46a6f61c4fd34" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45687 } } peers { permanent_uuid: "efe5abfec33149a292aa7506ed2a0482" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40071 } }
I20260809 04:48:30.876688 564 leader_election.cc:288] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34 [CANDIDATE]: Term 1 election: Requested vote from peers 0f842d4a2ede4119b0e11288dc15f446 (127.12.234.1:41357), efe5abfec33149a292aa7506ed2a0482 (127.12.234.3:40071)
I20260809 04:48:30.877178 485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "072854d1b25d44c2a4dbcdaefa1c7579" candidate_uuid: "22b8c9a67ba14225a4d46a6f61c4fd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe5abfec33149a292aa7506ed2a0482"
I20260809 04:48:30.877163 32667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "072854d1b25d44c2a4dbcdaefa1c7579" candidate_uuid: "22b8c9a67ba14225a4d46a6f61c4fd34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f842d4a2ede4119b0e11288dc15f446"
I20260809 04:48:30.877381 485 raft_consensus.cc:3037] T 072854d1b25d44c2a4dbcdaefa1c7579 P efe5abfec33149a292aa7506ed2a0482 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:30.877393 32667 raft_consensus.cc:3037] T 072854d1b25d44c2a4dbcdaefa1c7579 P 0f842d4a2ede4119b0e11288dc15f446 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:30.880698 485 raft_consensus.cc:2454] T 072854d1b25d44c2a4dbcdaefa1c7579 P efe5abfec33149a292aa7506ed2a0482 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22b8c9a67ba14225a4d46a6f61c4fd34 in term 1.
I20260809 04:48:30.880698 32667 raft_consensus.cc:2454] T 072854d1b25d44c2a4dbcdaefa1c7579 P 0f842d4a2ede4119b0e11288dc15f446 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22b8c9a67ba14225a4d46a6f61c4fd34 in term 1.
I20260809 04:48:30.881188 32739 leader_election.cc:302] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0f842d4a2ede4119b0e11288dc15f446, 22b8c9a67ba14225a4d46a6f61c4fd34, efe5abfec33149a292aa7506ed2a0482; no voters:
I20260809 04:48:30.881585 564 raft_consensus.cc:2781] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:48:30.882057 564 raft_consensus.cc:686] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34 [term 1 LEADER]: Becoming Leader. State: Replica: 22b8c9a67ba14225a4d46a6f61c4fd34, State: Running, Role: LEADER
I20260809 04:48:30.882588 564 consensus_queue.cc:237] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f842d4a2ede4119b0e11288dc15f446" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 41357 } } peers { permanent_uuid: "22b8c9a67ba14225a4d46a6f61c4fd34" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45687 } } peers { permanent_uuid: "efe5abfec33149a292aa7506ed2a0482" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40071 } }
I20260809 04:48:30.887027 32512 catalog_manager.cc:5495] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34 reported cstate change: term changed from 0 to 1, leader changed from <none> to 22b8c9a67ba14225a4d46a6f61c4fd34 (127.12.234.2). New cstate: current_term: 1 leader_uuid: "22b8c9a67ba14225a4d46a6f61c4fd34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f842d4a2ede4119b0e11288dc15f446" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 41357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22b8c9a67ba14225a4d46a6f61c4fd34" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efe5abfec33149a292aa7506ed2a0482" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40071 } health_report { overall_health: UNKNOWN } } }
I20260809 04:48:30.915602 554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:30.909513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58374 (local address 127.12.234.2:45687)
0809 04:48:30.909797 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:30.909804 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:30.909857 (+ 53us) server_negotiation.cc:407] Connection header received
0809 04:48:30.910127 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:30.910132 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:30.910246 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:30.910424 (+ 178us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:48:30.910729 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.911569 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.913391 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.913868 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.913986 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:30.914431 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:48:30.914941 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:48:30.915064 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:30.915152 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 04:48:30.915221 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:48:30.923861 485 raft_consensus.cc:1261] T 072854d1b25d44c2a4dbcdaefa1c7579 P efe5abfec33149a292aa7506ed2a0482 [term 1 FOLLOWER]: Refusing update from remote peer 22b8c9a67ba14225a4d46a6f61c4fd34: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:48:30.923861 32667 raft_consensus.cc:1261] T 072854d1b25d44c2a4dbcdaefa1c7579 P 0f842d4a2ede4119b0e11288dc15f446 [term 1 FOLLOWER]: Refusing update from remote peer 22b8c9a67ba14225a4d46a6f61c4fd34: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:48:30.924988 567 consensus_queue.cc:1035] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe5abfec33149a292aa7506ed2a0482" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 40071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:48:30.925369 564 consensus_queue.cc:1035] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f842d4a2ede4119b0e11288dc15f446" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 41357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:48:30.937356 570 mvcc.cc:205] Tried to move back new op lower bound from 7316483731133153280 to 7316483730979291136. Current Snapshot: MvccSnapshot[applied={T|T < 7316483731133153280}]
I20260809 04:48:30.940910 571 mvcc.cc:205] Tried to move back new op lower bound from 7316483731133153280 to 7316483730979291136. Current Snapshot: MvccSnapshot[applied={T|T < 7316483731133153280 or (T in {7316483731133153280})}]
I20260809 04:48:30.948714 32513 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:48:30.948990 32513 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:48:30.952432 32513 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 60c5c4bcbc5c49c090272ba7eec48636: Sending DeleteTablet for 3 replicas of tablet 072854d1b25d44c2a4dbcdaefa1c7579
I20260809 04:48:30.953518 32646 tablet_service.cc:1514] Processing DeleteTablet for tablet 072854d1b25d44c2a4dbcdaefa1c7579 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:48:30 UTC) from {username='slave'} at 127.0.0.1:38692
I20260809 04:48:30.953704 461 tablet_service.cc:1514] Processing DeleteTablet for tablet 072854d1b25d44c2a4dbcdaefa1c7579 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:48:30 UTC) from {username='slave'} at 127.0.0.1:52670
I20260809 04:48:30.954345 322 tablet_service.cc:1514] Processing DeleteTablet for tablet 072854d1b25d44c2a4dbcdaefa1c7579 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:48:30 UTC) from {username='slave'} at 127.0.0.1:58362
I20260809 04:48:30.955178 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 32580
I20260809 04:48:30.972412 582 tablet_replica.cc:330] T 072854d1b25d44c2a4dbcdaefa1c7579 P efe5abfec33149a292aa7506ed2a0482: stopping tablet replica
I20260809 04:48:30.973042 583 tablet_replica.cc:330] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34: stopping tablet replica
I20260809 04:48:30.973208 581 tablet_replica.cc:330] T 072854d1b25d44c2a4dbcdaefa1c7579 P 0f842d4a2ede4119b0e11288dc15f446: stopping tablet replica
I20260809 04:48:30.973464 583 raft_consensus.cc:2229] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:48:30.973562 581 raft_consensus.cc:2229] T 072854d1b25d44c2a4dbcdaefa1c7579 P 0f842d4a2ede4119b0e11288dc15f446 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:48:30.974041 581 raft_consensus.cc:2258] T 072854d1b25d44c2a4dbcdaefa1c7579 P 0f842d4a2ede4119b0e11288dc15f446 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:48:30.974079 583 raft_consensus.cc:2258] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:48:30.974637 582 raft_consensus.cc:2229] T 072854d1b25d44c2a4dbcdaefa1c7579 P efe5abfec33149a292aa7506ed2a0482 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:48:30.975059 582 raft_consensus.cc:2258] T 072854d1b25d44c2a4dbcdaefa1c7579 P efe5abfec33149a292aa7506ed2a0482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:48:30.977106 581 ts_tablet_manager.cc:1905] T 072854d1b25d44c2a4dbcdaefa1c7579 P 0f842d4a2ede4119b0e11288dc15f446: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:48:30.977157 582 ts_tablet_manager.cc:1905] T 072854d1b25d44c2a4dbcdaefa1c7579 P efe5abfec33149a292aa7506ed2a0482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:48:30.978272 550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:30.958210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38702 (local address 127.12.234.1:41357)
0809 04:48:30.958415 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:30.958423 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:30.966678 (+ 8255us) server_negotiation.cc:407] Connection header received
0809 04:48:30.967132 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:30.967140 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:30.967245 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:30.967328 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:30.968171 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.969631 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.974452 (+ 4821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:30.975030 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:30.975132 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:30.976640 (+ 1508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:30.976650 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:30.976666 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:30.977368 (+ 702us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:30.977428 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:30.977451 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:30.977540 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:30.977804 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 04:48:30.977919 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 04:48:30.979327 32710 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:30.985598 582 ts_tablet_manager.cc:1918] T 072854d1b25d44c2a4dbcdaefa1c7579 P efe5abfec33149a292aa7506ed2a0482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:48:30.985795 582 log.cc:1196] T 072854d1b25d44c2a4dbcdaefa1c7579 P efe5abfec33149a292aa7506ed2a0482: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-2/wal/wals/072854d1b25d44c2a4dbcdaefa1c7579
I20260809 04:48:30.987008 582 ts_tablet_manager.cc:1939] T 072854d1b25d44c2a4dbcdaefa1c7579 P efe5abfec33149a292aa7506ed2a0482: Deleting consensus metadata
I20260809 04:48:30.989142 583 ts_tablet_manager.cc:1905] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:48:30.991048 32497 catalog_manager.cc:4841] TS efe5abfec33149a292aa7506ed2a0482 (127.12.234.3:40071): tablet 072854d1b25d44c2a4dbcdaefa1c7579 (table test-table [id=b27c5b3bbded4637aa51008d1eb70fa1]) successfully deleted
I20260809 04:48:30.996706 583 ts_tablet_manager.cc:1918] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:48:30.996866 583 log.cc:1196] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786250603326224-13224-0/minicluster-data/ts-1/wal/wals/072854d1b25d44c2a4dbcdaefa1c7579
I20260809 04:48:30.997468 583 ts_tablet_manager.cc:1939] T 072854d1b25d44c2a4dbcdaefa1c7579 P 22b8c9a67ba14225a4d46a6f61c4fd34: Deleting consensus metadata
I20260809 04:48:30.999178 32494 catalog_manager.cc:4841] TS 22b8c9a67ba14225a4d46a6f61c4fd34 (127.12.234.2:45687): tablet 072854d1b25d44c2a4dbcdaefa1c7579 (table test-table [id=b27c5b3bbded4637aa51008d1eb70fa1]) successfully deleted
I20260809 04:48:31.073272 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 32580
I20260809 04:48:31.095650 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 32719
W20260809 04:48:31.095674 32494 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.234.1:41357 (error 108)
I20260809 04:48:31.097988 551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:31.097442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:41357 (local address 127.0.0.1:38710)
0809 04:48:31.097682 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:31.097809 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260809 04:48:31.098239 32494 catalog_manager.cc:4584] TS 0f842d4a2ede4119b0e11288dc15f446 (127.12.234.1:41357): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 072854d1b25d44c2a4dbcdaefa1c7579: Network error: Client connection negotiation failed: client connection to 127.12.234.1:41357: connect: Connection refused (error 111)
I20260809 04:48:31.103578 554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:31.096962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58380 (local address 127.12.234.2:45687)
0809 04:48:31.097188 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:31.097194 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:31.097212 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:48:31.097274 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:31.097278 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:31.097350 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:31.097484 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:31.097889 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:31.098894 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:31.100213 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:31.100874 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:31.101001 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:31.101974 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:31.101984 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:31.102002 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:31.102757 (+ 755us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:31.102820 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:31.102844 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:31.102959 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:31.103101 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 04:48:31.103206 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 04:48:31.104481 387 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:31.151798 551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:31.151329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:41357 (local address 127.0.0.1:38720)
0809 04:48:31.151558 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:31.151643 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260809 04:48:31.152904 551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:31.152494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:41357 (local address 127.0.0.1:38734)
0809 04:48:31.152702 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:31.152769 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 04:48:31.213838 551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:31.213268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:41357 (local address 127.0.0.1:38750)
0809 04:48:31.213473 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:31.213629 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 04:48:31.213974 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 32719
I20260809 04:48:31.220638 551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:31.220291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:41357 (local address 127.0.0.1:38760)
0809 04:48:31.220445 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:31.220517 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 04:48:31.237762 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 397
I20260809 04:48:31.244930 553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:31.238540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52680 (local address 127.12.234.3:40071)
0809 04:48:31.238717 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:31.238722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:31.238754 (+ 32us) server_negotiation.cc:407] Connection header received
0809 04:48:31.238927 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:31.238931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:31.239037 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:31.239117 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:31.239555 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:31.240315 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:31.242114 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:31.242473 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:31.242567 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:31.243659 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:31.243667 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:31.243681 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:31.244269 (+ 588us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:31.244319 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:31.244337 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:31.244415 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:31.244557 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 04:48:31.244634 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 04:48:31.245486 528 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:31.301834 551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:31.301373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:41357 (local address 127.0.0.1:60520)
0809 04:48:31.301586 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:31.301673 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 04:48:31.303012 551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:31.302610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:41357 (local address 127.0.0.1:60522)
0809 04:48:31.302815 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:31.302886 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:41357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260809 04:48:31.334739 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 397
I20260809 04:48:31.355216 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 32479
I20260809 04:48:31.356415 32546 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:31.465345 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 32479
2026-08-09T04:48:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2749 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:48:31 dist-test-slave-1jjb krb5kdc[595](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:48:31 dist-test-slave-1jjb krb5kdc[595](info): set up 2 sockets
Aug 09 04:48:31 dist-test-slave-1jjb krb5kdc[595](info): commencing operation
krb5kdc: starting...
W20260809 04:48:32.605927 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:48:32 dist-test-slave-1jjb krb5kdc[595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:48:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:48:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:48:32.861052 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:33435
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:33435
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:48:33.165235 610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:33.165458 610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:33.165583 610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:33.172061 610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:48:33.172158 610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:33.172261 610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:33.172359 610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:48:33.172434 610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:48:33.180646 610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:33435
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:33435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:33.181819 610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:33.182883 610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:33.193679 617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:33.193761 619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:33.193761 616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:33.194725 610 server_base.cc:1034] running on GCE node
I20260809 04:48:33.195452 610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:33.196749 610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:33.197994 610 hybrid_clock.cc:648] HybridClock initialized: now 1786250913197948 us; error 50 us; skew 500 ppm
I20260809 04:48:33.198328 610 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:48:33 dist-test-slave-1jjb krb5kdc[595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250913, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:33.201330 610 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:48:33.202572 610 webserver.cc:460] Webserver started at http://127.12.234.62:43555/ using document root <none> and password file <none>
I20260809 04:48:33.203145 610 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:33.203243 610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:33.203483 610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:33.206219 610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "0da19afd0b9b48b0a0646c2b69cb5c3e"
format_stamp: "Formatted at 2026-08-09 04:48:33 on dist-test-slave-1jjb"
server_key: "d4dddd7a8114d9c92a668267e80bcb7a"
server_key_iv: "03e8f67ad6a2dd63e7c67d163b1eca32"
server_key_version: "encryptionkey@0"
I20260809 04:48:33.206840 610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "0da19afd0b9b48b0a0646c2b69cb5c3e"
format_stamp: "Formatted at 2026-08-09 04:48:33 on dist-test-slave-1jjb"
server_key: "d4dddd7a8114d9c92a668267e80bcb7a"
server_key_iv: "03e8f67ad6a2dd63e7c67d163b1eca32"
server_key_version: "encryptionkey@0"
I20260809 04:48:33.211644 610 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 04:48:33.214710 627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:33.215817 610 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 04:48:33.215994 610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "0da19afd0b9b48b0a0646c2b69cb5c3e"
format_stamp: "Formatted at 2026-08-09 04:48:33 on dist-test-slave-1jjb"
server_key: "d4dddd7a8114d9c92a668267e80bcb7a"
server_key_iv: "03e8f67ad6a2dd63e7c67d163b1eca32"
server_key_version: "encryptionkey@0"
I20260809 04:48:33.216239 610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:33.227916 610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:33.231472 610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:33.231732 610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:33.248960 610 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:33435
I20260809 04:48:33.248994 686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:33435 every 8 connection(s)
I20260809 04:48:33.250216 610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:48:33.251807 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 610
I20260809 04:48:33.252007 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:48:33.252559 13224 external_mini_cluster.cc:1428] Setting key fef7f750ab3ef3e3004ca84dc221e150
I20260809 04:48:33.254352 687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 04:48:33 dist-test-slave-1jjb krb5kdc[595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:48:33.266719 687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0da19afd0b9b48b0a0646c2b69cb5c3e: Bootstrap starting.
I20260809 04:48:33.270720 687 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0da19afd0b9b48b0a0646c2b69cb5c3e: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:33.271898 687 log.cc:824] T 00000000000000000000000000000000 P 0da19afd0b9b48b0a0646c2b69cb5c3e: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:33.274591 687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0da19afd0b9b48b0a0646c2b69cb5c3e: No bootstrap required, opened a new log
I20260809 04:48:33.274816 690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:33.254203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35040 (local address 127.12.234.62:33435)
0809 04:48:33.254967 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:33.254977 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:33.255013 (+ 36us) server_negotiation.cc:407] Connection header received
0809 04:48:33.256579 (+ 1566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:33.256597 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:33.256906 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:33.257198 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:33.258982 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:33.260231 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:33.261868 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:33.262467 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:33.262590 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:33.267030 (+ 4440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:33.267042 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:33.267058 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:33.267105 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:33.270880 (+ 3775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:33.271551 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:33.271565 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:33.271576 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:33.271788 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:33.272252 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:33.272257 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:33.272260 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:33.272872 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:33.273161 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:33.273614 (+ 453us) server_negotiation.cc:299] Negotiation successful
0809 04:48:33.273962 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":396,"threads_started":1}
I20260809 04:48:33.282159 687 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0da19afd0b9b48b0a0646c2b69cb5c3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da19afd0b9b48b0a0646c2b69cb5c3e" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 33435 } }
I20260809 04:48:33.282505 687 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0da19afd0b9b48b0a0646c2b69cb5c3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:33.282611 687 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0da19afd0b9b48b0a0646c2b69cb5c3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0da19afd0b9b48b0a0646c2b69cb5c3e, State: Initialized, Role: FOLLOWER
I20260809 04:48:33.283213 687 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0da19afd0b9b48b0a0646c2b69cb5c3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da19afd0b9b48b0a0646c2b69cb5c3e" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 33435 } }
I20260809 04:48:33.283488 687 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0da19afd0b9b48b0a0646c2b69cb5c3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:48:33.283634 687 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0da19afd0b9b48b0a0646c2b69cb5c3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:48:33.283807 687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0da19afd0b9b48b0a0646c2b69cb5c3e [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:33.285750 687 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0da19afd0b9b48b0a0646c2b69cb5c3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da19afd0b9b48b0a0646c2b69cb5c3e" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 33435 } }
I20260809 04:48:33.286185 687 leader_election.cc:302] T 00000000000000000000000000000000 P 0da19afd0b9b48b0a0646c2b69cb5c3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0da19afd0b9b48b0a0646c2b69cb5c3e; no voters:
I20260809 04:48:33.286720 687 leader_election.cc:288] T 00000000000000000000000000000000 P 0da19afd0b9b48b0a0646c2b69cb5c3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:48:33.286864 692 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0da19afd0b9b48b0a0646c2b69cb5c3e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:48:33.288265 692 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0da19afd0b9b48b0a0646c2b69cb5c3e [term 1 LEADER]: Becoming Leader. State: Replica: 0da19afd0b9b48b0a0646c2b69cb5c3e, State: Running, Role: LEADER
I20260809 04:48:33.288763 687 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0da19afd0b9b48b0a0646c2b69cb5c3e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:48:33.288733 692 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0da19afd0b9b48b0a0646c2b69cb5c3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da19afd0b9b48b0a0646c2b69cb5c3e" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 33435 } }
I20260809 04:48:33.291301 694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0da19afd0b9b48b0a0646c2b69cb5c3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0da19afd0b9b48b0a0646c2b69cb5c3e. Latest consensus state: current_term: 1 leader_uuid: "0da19afd0b9b48b0a0646c2b69cb5c3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da19afd0b9b48b0a0646c2b69cb5c3e" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 33435 } } }
I20260809 04:48:33.291674 694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0da19afd0b9b48b0a0646c2b69cb5c3e [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:33.292042 693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0da19afd0b9b48b0a0646c2b69cb5c3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0da19afd0b9b48b0a0646c2b69cb5c3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da19afd0b9b48b0a0646c2b69cb5c3e" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 33435 } } }
I20260809 04:48:33.292465 693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0da19afd0b9b48b0a0646c2b69cb5c3e [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:33.297248 700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:48:33.302181 700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:48:33.310462 700 catalog_manager.cc:1347] Generated new cluster ID: ecb51f024b994e6687e495c199733870
I20260809 04:48:33.310565 700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:48:33.321880 700 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:48:33.322992 700 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:48:33.334263 700 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0da19afd0b9b48b0a0646c2b69cb5c3e: Generated new TSK 0
I20260809 04:48:33.334991 700 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
I20260809 04:48:33.420884 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:0
--local_ip_for_outbound_sockets=127.12.234.1
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:33435
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:48:33.749269 715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:33.749529 715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:33.749679 715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:33.755656 715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:33.755736 715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:33.755924 715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:48:33.764357 715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:33435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:33.765375 715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:33.766507 715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:33.778352 722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:33.778398 723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:33.778312 725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:33.783457 715 server_base.cc:1034] running on GCE node
I20260809 04:48:33.784011 715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:33.784951 715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:33.786162 715 hybrid_clock.cc:648] HybridClock initialized: now 1786250913786113 us; error 54 us; skew 500 ppm
I20260809 04:48:33.786502 715 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:48:33 dist-test-slave-1jjb krb5kdc[595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250913, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:33.789907 715 init.cc:377] Logged in from keytab as kudu/127.12.234.1@KRBTEST.COM (short username kudu)
I20260809 04:48:33.791373 715 webserver.cc:460] Webserver started at http://127.12.234.1:37461/ using document root <none> and password file <none>
I20260809 04:48:33.791978 715 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:33.792142 715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:33.792407 715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:33.795022 715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-0/data/instance:
uuid: "b9537463e92740d096661779d87c8baa"
format_stamp: "Formatted at 2026-08-09 04:48:33 on dist-test-slave-1jjb"
server_key: "48544c053456503eee28b1ecc7f8fa7b"
server_key_iv: "40a744c303a31f06daefb2d57fadb500"
server_key_version: "encryptionkey@0"
I20260809 04:48:33.795646 715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance:
uuid: "b9537463e92740d096661779d87c8baa"
format_stamp: "Formatted at 2026-08-09 04:48:33 on dist-test-slave-1jjb"
server_key: "48544c053456503eee28b1ecc7f8fa7b"
server_key_iv: "40a744c303a31f06daefb2d57fadb500"
server_key_version: "encryptionkey@0"
I20260809 04:48:33.800501 715 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 04:48:33.803541 733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:33.804700 715 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:48:33.804857 715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "b9537463e92740d096661779d87c8baa"
format_stamp: "Formatted at 2026-08-09 04:48:33 on dist-test-slave-1jjb"
server_key: "48544c053456503eee28b1ecc7f8fa7b"
server_key_iv: "40a744c303a31f06daefb2d57fadb500"
server_key_version: "encryptionkey@0"
I20260809 04:48:33.805028 715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:33.822804 715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:33.826757 715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:33.827009 715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:33.827812 715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:33.829137 715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:48:33.829208 715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:33.829361 715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:33.829417 715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:33.860747 715 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:38657
I20260809 04:48:33.860884 858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:38657 every 8 connection(s)
I20260809 04:48:33.861876 715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:48:33.869062 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 715
I20260809 04:48:33.869236 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance
I20260809 04:48:33.869673 13224 external_mini_cluster.cc:1428] Setting key 627e662f1e7c7a14c4029bc6edd2d051
Aug 09 04:48:33 dist-test-slave-1jjb krb5kdc[595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250913, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:48:33.881042 861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:33.864164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:50375 (local address 127.12.234.62:33435)
0809 04:48:33.864615 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:33.864619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:33.865471 (+ 852us) server_negotiation.cc:407] Connection header received
0809 04:48:33.867198 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:33.867203 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:33.867286 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:33.867410 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:33.868960 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:33.869911 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:33.871686 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:33.872059 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:33.872234 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:33.875542 (+ 3308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:33.875547 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:33.875550 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:33.875579 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:33.877849 (+ 2270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:33.878644 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:33.878649 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:33.878651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:33.878777 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:33.879268 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:33.879275 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:33.879278 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:33.879535 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:33.879687 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:33.880418 (+ 731us) server_negotiation.cc:299] Negotiation successful
0809 04:48:33.880629 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":233,"threads_started":1}
I20260809 04:48:33.881811 862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:33.864658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:33435 (local address 127.12.234.1:50375)
0809 04:48:33.865313 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:33.865355 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:33.866946 (+ 1591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:33.867641 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:33.867648 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:33.868229 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:48:33.868767 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:33.868785 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:33.870088 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:33.870092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:33.871537 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:33.871549 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:33.872313 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:33.872320 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:33.872543 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:33.873172 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:48:33.873199 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:48:33.875338 (+ 2139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:48:33.878012 (+ 2674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:33.878021 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:33.878035 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:33.878481 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:33.878910 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:33.878916 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:33.878920 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:33.879125 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:33.879674 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:48:33.879682 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:48:33.879999 (+ 317us) client_negotiation.cc:769] Sending connection context
0809 04:48:33.880322 (+ 323us) client_negotiation.cc:240] Negotiation successful
0809 04:48:33.880767 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":262,"threads_started":1}
I20260809 04:48:33.883352 859 heartbeater.cc:347] Connected to a master server at 127.12.234.62:33435
I20260809 04:48:33.883635 859 heartbeater.cc:464] Registering TS with master...
I20260809 04:48:33.884377 859 heartbeater.cc:511] Master 127.12.234.62:33435 requested a full tablet report, sending...
I20260809 04:48:33.886328 647 ts_manager.cc:194] Registered new tserver with Master: b9537463e92740d096661779d87c8baa (127.12.234.1:38657)
I20260809 04:48:33.887552 647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.1@KRBTEST.COM'} at 127.12.234.1:50375
WARNING: no policy specified for kudu/127.12.234.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
I20260809 04:48:33.931236 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:0
--local_ip_for_outbound_sockets=127.12.234.2
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:33435
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:48:34.239387 865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:34.239643 865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:34.239789 865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:34.245980 865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:34.246097 865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:34.246254 865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:48:34.254577 865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:33435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:34.255540 865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:34.256680 865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:34.268877 870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:34.269006 871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:34.269500 873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:34.269927 865 server_base.cc:1034] running on GCE node
I20260809 04:48:34.270495 865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:34.271345 865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:34.272575 865 hybrid_clock.cc:648] HybridClock initialized: now 1786250914272543 us; error 41 us; skew 500 ppm
I20260809 04:48:34.272850 865 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:48:34 dist-test-slave-1jjb krb5kdc[595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250914, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:34.280164 865 init.cc:377] Logged in from keytab as kudu/127.12.234.2@KRBTEST.COM (short username kudu)
I20260809 04:48:34.281397 865 webserver.cc:460] Webserver started at http://127.12.234.2:33877/ using document root <none> and password file <none>
I20260809 04:48:34.281940 865 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:34.282033 865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:34.282294 865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:34.284852 865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-1/data/instance:
uuid: "3438d9c22fe24859bbbad205af980f1f"
format_stamp: "Formatted at 2026-08-09 04:48:34 on dist-test-slave-1jjb"
server_key: "4b5bae0e9625ad6267729558cdf7fd3b"
server_key_iv: "694c64a35bcf6e21338b34bb16f48f73"
server_key_version: "encryptionkey@0"
I20260809 04:48:34.285447 865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance:
uuid: "3438d9c22fe24859bbbad205af980f1f"
format_stamp: "Formatted at 2026-08-09 04:48:34 on dist-test-slave-1jjb"
server_key: "4b5bae0e9625ad6267729558cdf7fd3b"
server_key_iv: "694c64a35bcf6e21338b34bb16f48f73"
server_key_version: "encryptionkey@0"
I20260809 04:48:34.290235 865 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:48:34.293499 881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:34.294646 865 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:48:34.294796 865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "3438d9c22fe24859bbbad205af980f1f"
format_stamp: "Formatted at 2026-08-09 04:48:34 on dist-test-slave-1jjb"
server_key: "4b5bae0e9625ad6267729558cdf7fd3b"
server_key_iv: "694c64a35bcf6e21338b34bb16f48f73"
server_key_version: "encryptionkey@0"
I20260809 04:48:34.294960 865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:34.316567 865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:34.320572 865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:34.320813 865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:34.321676 865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:34.322857 865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:48:34.322924 865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:34.323082 865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:34.323132 865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:34.353354 865 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:33095
I20260809 04:48:34.353426 1008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:33095 every 8 connection(s)
I20260809 04:48:34.354481 865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:48:34.364636 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 865
I20260809 04:48:34.364845 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance
I20260809 04:48:34.365324 13224 external_mini_cluster.cc:1428] Setting key 61718424bc0f87484d58bf72e7ddd711
Aug 09 04:48:34 dist-test-slave-1jjb krb5kdc[595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250914, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:48:34.373868 861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:34.356788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:37641 (local address 127.12.234.62:33435)
0809 04:48:34.356965 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:34.356969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:34.357957 (+ 988us) server_negotiation.cc:407] Connection header received
0809 04:48:34.359697 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:34.359702 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:34.359769 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:34.359865 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:34.361266 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.361949 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.363495 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.363868 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.363954 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:34.367247 (+ 3293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:34.367251 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:34.367254 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:34.367281 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:34.370900 (+ 3619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:34.371541 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:34.371548 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:34.371554 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:34.371705 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:34.372111 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:34.372117 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:34.372122 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:34.372472 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:34.372624 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:34.373169 (+ 545us) server_negotiation.cc:299] Negotiation successful
0809 04:48:34.373410 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 04:48:34.374521 1011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:34.357201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:33435 (local address 127.12.234.2:37641)
0809 04:48:34.357810 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:34.357842 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:34.359458 (+ 1616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:34.360050 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:34.360056 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:34.360595 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:48:34.361115 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:34.361131 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.362067 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.362069 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:34.363378 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:34.363390 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.364041 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.364045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:34.364266 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:34.364947 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:48:34.364976 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:48:34.367048 (+ 2072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:48:34.371028 (+ 3980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:34.371034 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:34.371044 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:34.371421 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:34.371807 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:34.371810 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:34.371812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:34.371981 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:34.372602 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:48:34.372607 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:48:34.372847 (+ 240us) client_negotiation.cc:769] Sending connection context
0809 04:48:34.373048 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 04:48:34.373363 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":238,"threads_started":1}
I20260809 04:48:34.375800 1009 heartbeater.cc:347] Connected to a master server at 127.12.234.62:33435
I20260809 04:48:34.376149 1009 heartbeater.cc:464] Registering TS with master...
I20260809 04:48:34.376796 1009 heartbeater.cc:511] Master 127.12.234.62:33435 requested a full tablet report, sending...
I20260809 04:48:34.378064 647 ts_manager.cc:194] Registered new tserver with Master: 3438d9c22fe24859bbbad205af980f1f (127.12.234.2:33095)
I20260809 04:48:34.378859 647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.2@KRBTEST.COM'} at 127.12.234.2:37641
WARNING: no policy specified for kudu/127.12.234.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
I20260809 04:48:34.425215 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:0
--local_ip_for_outbound_sockets=127.12.234.3
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:33435
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:33925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:48:34.730553 1014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:34.730775 1014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:34.730888 1014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:34.737299 1014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:34.737406 1014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:34.737567 1014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:48:34.745754 1014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:33925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:33435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:34.746753 1014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:34.747884 1014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:34.759852 1020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:34.759927 1025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:34.759873 1022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:34.760624 1014 server_base.cc:1034] running on GCE node
I20260809 04:48:34.761164 1014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:34.761981 1014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:34.763217 1014 hybrid_clock.cc:648] HybridClock initialized: now 1786250914763162 us; error 33 us; skew 500 ppm
I20260809 04:48:34.763590 1014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:48:34 dist-test-slave-1jjb krb5kdc[595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250914, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:34.766403 1014 init.cc:377] Logged in from keytab as kudu/127.12.234.3@KRBTEST.COM (short username kudu)
I20260809 04:48:34.767653 1014 webserver.cc:460] Webserver started at http://127.12.234.3:39417/ using document root <none> and password file <none>
I20260809 04:48:34.768381 1014 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:34.768524 1014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:34.768826 1014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:34.771412 1014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-2/data/instance:
uuid: "f6621dcc3ce94c0cb8e5ce9b7ad59c8e"
format_stamp: "Formatted at 2026-08-09 04:48:34 on dist-test-slave-1jjb"
server_key: "e52fa41ab6164c28047cadc2829410f4"
server_key_iv: "cc3d570e1dfbac7638ea3f18c74b7f14"
server_key_version: "encryptionkey@0"
I20260809 04:48:34.772161 1014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance:
uuid: "f6621dcc3ce94c0cb8e5ce9b7ad59c8e"
format_stamp: "Formatted at 2026-08-09 04:48:34 on dist-test-slave-1jjb"
server_key: "e52fa41ab6164c28047cadc2829410f4"
server_key_iv: "cc3d570e1dfbac7638ea3f18c74b7f14"
server_key_version: "encryptionkey@0"
I20260809 04:48:34.776746 1014 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260809 04:48:34.779697 1035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:34.780795 1014 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:48:34.780948 1014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "f6621dcc3ce94c0cb8e5ce9b7ad59c8e"
format_stamp: "Formatted at 2026-08-09 04:48:34 on dist-test-slave-1jjb"
server_key: "e52fa41ab6164c28047cadc2829410f4"
server_key_iv: "cc3d570e1dfbac7638ea3f18c74b7f14"
server_key_version: "encryptionkey@0"
I20260809 04:48:34.781136 1014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:34.799592 1014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:34.803297 1014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:34.803555 1014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:34.804473 1014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:34.805740 1014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:48:34.805837 1014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:34.805971 1014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:34.806039 1014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:34.837908 1014 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:39553
I20260809 04:48:34.837973 1159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:39553 every 8 connection(s)
I20260809 04:48:34.839116 1014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:48:34.846953 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 1014
I20260809 04:48:34.847126 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance
I20260809 04:48:34.847572 13224 external_mini_cluster.cc:1428] Setting key cf058e309c3c66022e5687e8a8be3ade
Aug 09 04:48:34 dist-test-slave-1jjb krb5kdc[595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250914, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:48:34.857033 861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:34.841405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:52241 (local address 127.12.234.62:33435)
0809 04:48:34.841612 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:34.841616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:34.842631 (+ 1015us) server_negotiation.cc:407] Connection header received
0809 04:48:34.844164 (+ 1533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:34.844168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:34.844230 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:34.844307 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:34.845509 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.846149 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.847503 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.848041 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.848220 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:34.852009 (+ 3789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:34.852013 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:34.852016 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:34.852047 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:34.854503 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:34.855126 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:34.855130 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:34.855132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:34.855252 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:34.855613 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:34.855616 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:34.855618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:34.855849 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:34.856012 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:34.856646 (+ 634us) server_negotiation.cc:299] Negotiation successful
0809 04:48:34.856802 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 04:48:34.857837 1162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:34.841838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:33435 (local address 127.12.234.3:52241)
0809 04:48:34.842475 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:34.842507 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:34.843904 (+ 1397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:34.844481 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:34.844488 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:34.844919 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:48:34.845370 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:34.845386 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.846267 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.846270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:34.847379 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:34.847389 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.848331 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.848338 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:34.848575 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:34.849395 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:48:34.849422 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:48:34.851818 (+ 2396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:48:34.854631 (+ 2813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:34.854637 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:34.854647 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:34.855029 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:34.855357 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:34.855365 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:34.855367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:34.855525 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:34.855961 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:48:34.855966 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:48:34.856293 (+ 327us) client_negotiation.cc:769] Sending connection context
0809 04:48:34.856531 (+ 238us) client_negotiation.cc:240] Negotiation successful
0809 04:48:34.856877 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":252,"threads_started":1}
I20260809 04:48:34.859045 1160 heartbeater.cc:347] Connected to a master server at 127.12.234.62:33435
I20260809 04:48:34.859350 1160 heartbeater.cc:464] Registering TS with master...
I20260809 04:48:34.859947 1160 heartbeater.cc:511] Master 127.12.234.62:33435 requested a full tablet report, sending...
I20260809 04:48:34.861451 647 ts_manager.cc:194] Registered new tserver with Master: f6621dcc3ce94c0cb8e5ce9b7ad59c8e (127.12.234.3:39553)
I20260809 04:48:34.862342 647 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.3@KRBTEST.COM'} at 127.12.234.3:52241
I20260809 04:48:34.862732 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:48:34.878759 861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:34.867338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35042 (local address 127.12.234.62:33435)
0809 04:48:34.867511 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:34.867519 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:34.867865 (+ 346us) server_negotiation.cc:407] Connection header received
0809 04:48:34.867969 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:34.867972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:34.868018 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:34.868164 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:34.869483 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.870127 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.871293 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.871644 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.871723 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:34.874148 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:34.874153 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:34.874156 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:34.874182 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:34.876532 (+ 2350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:34.877116 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:34.877119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:34.877121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:34.877221 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:34.877575 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:34.877578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:34.877580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:34.877814 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:34.877991 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:34.878309 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 04:48:34.878460 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:48:34.881975 647 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35042:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:48:34.887969 647 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:48:34.890991 859 heartbeater.cc:503] Master 127.12.234.62:33435 was elected leader, sending a full tablet report...
I20260809 04:48:34.918664 1173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:34.899133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:38657 (local address 127.0.0.1:53840)
0809 04:48:34.911263 (+ 12130us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:34.911276 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:34.911400 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:34.911897 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:34.911903 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:34.911940 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:34.912355 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:34.912368 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.914118 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.914124 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:34.916092 (+ 1968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:34.916108 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.917768 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.917775 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:34.917961 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:34.918011 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 04:48:34.918159 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 04:48:34.918323 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12022,"thread_start_us":11173,"threads_started":1}
I20260809 04:48:34.918701 1177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:34.899567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53840 (local address 127.12.234.1:38657)
0809 04:48:34.909459 (+ 9892us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:34.909469 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:34.911374 (+ 1905us) server_negotiation.cc:407] Connection header received
0809 04:48:34.911547 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:34.911554 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:34.911710 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:34.911878 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:34.912530 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.913964 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.916256 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.917549 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.917680 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:34.917917 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:34.918243 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 04:48:34.918402 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9696,"thread_start_us":9538,"threads_started":1}
I20260809 04:48:34.920948 1178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:34.900522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:33095 (local address 127.0.0.1:43500)
0809 04:48:34.912849 (+ 12327us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:34.912865 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:34.912965 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:34.913728 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:34.913733 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:34.913753 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:34.914081 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:34.914093 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.915775 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.915780 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:34.917722 (+ 1942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:34.917739 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.920191 (+ 2452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.920199 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:34.920366 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:34.920400 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 04:48:34.920495 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 04:48:34.920616 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12235,"thread_start_us":12117,"threads_started":1}
I20260809 04:48:34.920979 1179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:34.900973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43500 (local address 127.12.234.2:33095)
0809 04:48:34.912490 (+ 11517us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:34.912499 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:34.913083 (+ 584us) server_negotiation.cc:407] Connection header received
0809 04:48:34.913168 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:34.913174 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:34.913366 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:34.913524 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:34.914274 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.915601 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.917892 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.919958 (+ 2066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.920115 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:34.920303 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:34.920563 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 04:48:34.920707 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11344,"thread_start_us":11243,"threads_started":1}
I20260809 04:48:34.922456 787 tablet_service.cc:1467] Processing CreateTablet for tablet e30ff1c1b0934d28af6e6532b5ea3cff (DEFAULT_TABLE table=test-table [id=22c5730e457f45d0bca3d200596a5ea0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:48:34.923435 1174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:34.898993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:39553 (local address 127.0.0.1:52948)
0809 04:48:34.909840 (+ 10847us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:34.909884 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:34.910086 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:34.916975 (+ 6889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:34.916979 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:34.917003 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:34.917401 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:34.917411 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.919410 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.919417 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:34.921460 (+ 2043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:34.921472 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.922712 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.922719 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:34.922869 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:34.922903 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 04:48:34.923004 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 04:48:34.923121 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10558,"mutex_wait_us":3,"thread_start_us":12027,"threads_started":1}
I20260809 04:48:34.923476 1175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:34.899127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52948 (local address 127.12.234.3:39553)
0809 04:48:34.916538 (+ 17411us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:34.916543 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:34.916564 (+ 21us) server_negotiation.cc:407] Connection header received
0809 04:48:34.916625 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:34.916629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:34.916750 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:34.916981 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:34.917554 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.919251 (+ 1697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.921649 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.922514 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.922644 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:34.922866 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:34.923053 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 04:48:34.923201 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17264,"thread_start_us":17129,"threads_started":1}
I20260809 04:48:34.923820 787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e30ff1c1b0934d28af6e6532b5ea3cff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:34.924082 938 tablet_service.cc:1467] Processing CreateTablet for tablet e30ff1c1b0934d28af6e6532b5ea3cff (DEFAULT_TABLE table=test-table [id=22c5730e457f45d0bca3d200596a5ea0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:48:34.925292 938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e30ff1c1b0934d28af6e6532b5ea3cff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:34.925848 1088 tablet_service.cc:1467] Processing CreateTablet for tablet e30ff1c1b0934d28af6e6532b5ea3cff (DEFAULT_TABLE table=test-table [id=22c5730e457f45d0bca3d200596a5ea0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:48:34.926695 1088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e30ff1c1b0934d28af6e6532b5ea3cff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:34.936614 1180 tablet_bootstrap.cc:492] T e30ff1c1b0934d28af6e6532b5ea3cff P f6621dcc3ce94c0cb8e5ce9b7ad59c8e: Bootstrap starting.
I20260809 04:48:34.937976 1181 tablet_bootstrap.cc:492] T e30ff1c1b0934d28af6e6532b5ea3cff P 3438d9c22fe24859bbbad205af980f1f: Bootstrap starting.
I20260809 04:48:34.940445 1180 tablet_bootstrap.cc:654] T e30ff1c1b0934d28af6e6532b5ea3cff P f6621dcc3ce94c0cb8e5ce9b7ad59c8e: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:34.940745 1182 tablet_bootstrap.cc:492] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa: Bootstrap starting.
I20260809 04:48:34.941556 1181 tablet_bootstrap.cc:654] T e30ff1c1b0934d28af6e6532b5ea3cff P 3438d9c22fe24859bbbad205af980f1f: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:34.941793 1180 log.cc:824] T e30ff1c1b0934d28af6e6532b5ea3cff P f6621dcc3ce94c0cb8e5ce9b7ad59c8e: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:34.942620 1181 log.cc:824] T e30ff1c1b0934d28af6e6532b5ea3cff P 3438d9c22fe24859bbbad205af980f1f: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:34.943619 1182 tablet_bootstrap.cc:654] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:34.944666 1182 log.cc:824] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:34.945039 1180 tablet_bootstrap.cc:492] T e30ff1c1b0934d28af6e6532b5ea3cff P f6621dcc3ce94c0cb8e5ce9b7ad59c8e: No bootstrap required, opened a new log
I20260809 04:48:34.945344 1180 ts_tablet_manager.cc:1397] T e30ff1c1b0934d28af6e6532b5ea3cff P f6621dcc3ce94c0cb8e5ce9b7ad59c8e: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260809 04:48:34.947523 1182 tablet_bootstrap.cc:492] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa: No bootstrap required, opened a new log
I20260809 04:48:34.947551 1181 tablet_bootstrap.cc:492] T e30ff1c1b0934d28af6e6532b5ea3cff P 3438d9c22fe24859bbbad205af980f1f: No bootstrap required, opened a new log
I20260809 04:48:34.947755 1181 ts_tablet_manager.cc:1397] T e30ff1c1b0934d28af6e6532b5ea3cff P 3438d9c22fe24859bbbad205af980f1f: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 04:48:34.947760 1182 ts_tablet_manager.cc:1397] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 04:48:34.951937 1180 raft_consensus.cc:348] T e30ff1c1b0934d28af6e6532b5ea3cff P f6621dcc3ce94c0cb8e5ce9b7ad59c8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6621dcc3ce94c0cb8e5ce9b7ad59c8e" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 39553 } } peers { permanent_uuid: "b9537463e92740d096661779d87c8baa" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 38657 } } peers { permanent_uuid: "3438d9c22fe24859bbbad205af980f1f" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 33095 } }
I20260809 04:48:34.952359 1180 raft_consensus.cc:374] T e30ff1c1b0934d28af6e6532b5ea3cff P f6621dcc3ce94c0cb8e5ce9b7ad59c8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:34.952461 1180 raft_consensus.cc:729] T e30ff1c1b0934d28af6e6532b5ea3cff P f6621dcc3ce94c0cb8e5ce9b7ad59c8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6621dcc3ce94c0cb8e5ce9b7ad59c8e, State: Initialized, Role: FOLLOWER
I20260809 04:48:34.953040 1180 consensus_queue.cc:260] T e30ff1c1b0934d28af6e6532b5ea3cff P f6621dcc3ce94c0cb8e5ce9b7ad59c8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6621dcc3ce94c0cb8e5ce9b7ad59c8e" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 39553 } } peers { permanent_uuid: "b9537463e92740d096661779d87c8baa" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 38657 } } peers { permanent_uuid: "3438d9c22fe24859bbbad205af980f1f" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 33095 } }
I20260809 04:48:34.954031 1160 heartbeater.cc:503] Master 127.12.234.62:33435 was elected leader, sending a full tablet report...
I20260809 04:48:34.954664 1180 ts_tablet_manager.cc:1428] T e30ff1c1b0934d28af6e6532b5ea3cff P f6621dcc3ce94c0cb8e5ce9b7ad59c8e: Time spent starting tablet: real 0.009s user 0.003s sys 0.006s
I20260809 04:48:34.958237 1182 raft_consensus.cc:348] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6621dcc3ce94c0cb8e5ce9b7ad59c8e" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 39553 } } peers { permanent_uuid: "b9537463e92740d096661779d87c8baa" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 38657 } } peers { permanent_uuid: "3438d9c22fe24859bbbad205af980f1f" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 33095 } }
I20260809 04:48:34.958650 1182 raft_consensus.cc:374] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:34.958760 1182 raft_consensus.cc:729] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9537463e92740d096661779d87c8baa, State: Initialized, Role: FOLLOWER
I20260809 04:48:34.958734 1181 raft_consensus.cc:348] T e30ff1c1b0934d28af6e6532b5ea3cff P 3438d9c22fe24859bbbad205af980f1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6621dcc3ce94c0cb8e5ce9b7ad59c8e" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 39553 } } peers { permanent_uuid: "b9537463e92740d096661779d87c8baa" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 38657 } } peers { permanent_uuid: "3438d9c22fe24859bbbad205af980f1f" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 33095 } }
I20260809 04:48:34.959122 1181 raft_consensus.cc:374] T e30ff1c1b0934d28af6e6532b5ea3cff P 3438d9c22fe24859bbbad205af980f1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:34.959224 1181 raft_consensus.cc:729] T e30ff1c1b0934d28af6e6532b5ea3cff P 3438d9c22fe24859bbbad205af980f1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3438d9c22fe24859bbbad205af980f1f, State: Initialized, Role: FOLLOWER
I20260809 04:48:34.959353 1182 consensus_queue.cc:260] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6621dcc3ce94c0cb8e5ce9b7ad59c8e" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 39553 } } peers { permanent_uuid: "b9537463e92740d096661779d87c8baa" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 38657 } } peers { permanent_uuid: "3438d9c22fe24859bbbad205af980f1f" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 33095 } }
I20260809 04:48:34.959782 1181 consensus_queue.cc:260] T e30ff1c1b0934d28af6e6532b5ea3cff P 3438d9c22fe24859bbbad205af980f1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6621dcc3ce94c0cb8e5ce9b7ad59c8e" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 39553 } } peers { permanent_uuid: "b9537463e92740d096661779d87c8baa" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 38657 } } peers { permanent_uuid: "3438d9c22fe24859bbbad205af980f1f" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 33095 } }
I20260809 04:48:34.960911 1182 ts_tablet_manager.cc:1428] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260809 04:48:34.961230 1009 heartbeater.cc:503] Master 127.12.234.62:33435 was elected leader, sending a full tablet report...
I20260809 04:48:34.961293 1181 ts_tablet_manager.cc:1428] T e30ff1c1b0934d28af6e6532b5ea3cff P 3438d9c22fe24859bbbad205af980f1f: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260809 04:48:34.963585 1187 raft_consensus.cc:482] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:48:34.963812 1187 raft_consensus.cc:504] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6621dcc3ce94c0cb8e5ce9b7ad59c8e" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 39553 } } peers { permanent_uuid: "b9537463e92740d096661779d87c8baa" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 38657 } } peers { permanent_uuid: "3438d9c22fe24859bbbad205af980f1f" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 33095 } }
I20260809 04:48:34.965682 1187 leader_election.cc:288] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6621dcc3ce94c0cb8e5ce9b7ad59c8e (127.12.234.3:39553), 3438d9c22fe24859bbbad205af980f1f (127.12.234.2:33095)
I20260809 04:48:34.973654 1179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:34.966647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:36499 (local address 127.12.234.2:33095)
0809 04:48:34.966834 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:34.966839 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:34.967075 (+ 236us) server_negotiation.cc:407] Connection header received
0809 04:48:34.967185 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:34.967190 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:34.967255 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:34.967339 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:34.968014 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.968772 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.971493 (+ 2721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.972648 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.972766 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:34.972840 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:34.973285 (+ 445us) server_negotiation.cc:299] Negotiation successful
0809 04:48:34.973417 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 04:48:34.973654 1175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:34.965866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:50137 (local address 127.12.234.3:39553)
0809 04:48:34.966055 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:34.966061 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:34.966855 (+ 794us) server_negotiation.cc:407] Connection header received
0809 04:48:34.967125 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:34.967129 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:34.967197 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:34.967288 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:34.967947 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.968961 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.971486 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.972631 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.972775 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:34.972830 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:34.973285 (+ 455us) server_negotiation.cc:299] Negotiation successful
0809 04:48:34.973438 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 04:48:34.973928 1189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:34.966077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:33095 (local address 127.12.234.1:36499)
0809 04:48:34.966989 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:34.967004 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:34.967098 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:34.967345 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:34.967348 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:34.967362 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:34.967848 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:34.967858 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.968909 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.968914 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:34.971336 (+ 2422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:34.971350 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.972901 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.972905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:34.973113 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:34.973160 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 04:48:34.973248 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 04:48:34.973480 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"mutex_wait_us":157,"thread_start_us":348,"threads_started":1}
I20260809 04:48:34.974341 1190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:34.965787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:39553 (local address 127.12.234.1:50137)
0809 04:48:34.966731 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:34.966757 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:34.966959 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:34.967324 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:34.967330 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:34.967353 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:34.967750 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:34.967767 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.969095 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.969103 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:34.971337 (+ 2234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:34.971350 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:34.972879 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:34.972890 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:34.973126 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:34.973165 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 04:48:34.973248 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 04:48:34.973480 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":796,"mutex_wait_us":163,"thread_start_us":947,"threads_started":1}
I20260809 04:48:34.974879 961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e30ff1c1b0934d28af6e6532b5ea3cff" candidate_uuid: "b9537463e92740d096661779d87c8baa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3438d9c22fe24859bbbad205af980f1f" is_pre_election: true
I20260809 04:48:34.975178 1112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e30ff1c1b0934d28af6e6532b5ea3cff" candidate_uuid: "b9537463e92740d096661779d87c8baa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6621dcc3ce94c0cb8e5ce9b7ad59c8e" is_pre_election: true
I20260809 04:48:34.975303 961 raft_consensus.cc:2454] T e30ff1c1b0934d28af6e6532b5ea3cff P 3438d9c22fe24859bbbad205af980f1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9537463e92740d096661779d87c8baa in term 0.
I20260809 04:48:34.975508 1112 raft_consensus.cc:2454] T e30ff1c1b0934d28af6e6532b5ea3cff P f6621dcc3ce94c0cb8e5ce9b7ad59c8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9537463e92740d096661779d87c8baa in term 0.
I20260809 04:48:34.975965 738 leader_election.cc:302] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3438d9c22fe24859bbbad205af980f1f, b9537463e92740d096661779d87c8baa; no voters:
I20260809 04:48:34.976315 1187 raft_consensus.cc:2781] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:48:34.976451 1187 raft_consensus.cc:482] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:48:34.976554 1187 raft_consensus.cc:3037] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:34.978396 1187 raft_consensus.cc:504] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6621dcc3ce94c0cb8e5ce9b7ad59c8e" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 39553 } } peers { permanent_uuid: "b9537463e92740d096661779d87c8baa" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 38657 } } peers { permanent_uuid: "3438d9c22fe24859bbbad205af980f1f" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 33095 } }
I20260809 04:48:34.978921 1187 leader_election.cc:288] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa [CANDIDATE]: Term 1 election: Requested vote from peers f6621dcc3ce94c0cb8e5ce9b7ad59c8e (127.12.234.3:39553), 3438d9c22fe24859bbbad205af980f1f (127.12.234.2:33095)
I20260809 04:48:34.979322 1112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e30ff1c1b0934d28af6e6532b5ea3cff" candidate_uuid: "b9537463e92740d096661779d87c8baa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6621dcc3ce94c0cb8e5ce9b7ad59c8e"
I20260809 04:48:34.979374 961 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e30ff1c1b0934d28af6e6532b5ea3cff" candidate_uuid: "b9537463e92740d096661779d87c8baa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3438d9c22fe24859bbbad205af980f1f"
I20260809 04:48:34.979476 1112 raft_consensus.cc:3037] T e30ff1c1b0934d28af6e6532b5ea3cff P f6621dcc3ce94c0cb8e5ce9b7ad59c8e [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:34.979552 961 raft_consensus.cc:3037] T e30ff1c1b0934d28af6e6532b5ea3cff P 3438d9c22fe24859bbbad205af980f1f [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:34.981441 1112 raft_consensus.cc:2454] T e30ff1c1b0934d28af6e6532b5ea3cff P f6621dcc3ce94c0cb8e5ce9b7ad59c8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9537463e92740d096661779d87c8baa in term 1.
I20260809 04:48:34.981693 961 raft_consensus.cc:2454] T e30ff1c1b0934d28af6e6532b5ea3cff P 3438d9c22fe24859bbbad205af980f1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9537463e92740d096661779d87c8baa in term 1.
I20260809 04:48:34.981791 737 leader_election.cc:302] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b9537463e92740d096661779d87c8baa, f6621dcc3ce94c0cb8e5ce9b7ad59c8e; no voters:
I20260809 04:48:34.982084 1187 raft_consensus.cc:2781] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:48:34.982451 1187 raft_consensus.cc:686] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa [term 1 LEADER]: Becoming Leader. State: Replica: b9537463e92740d096661779d87c8baa, State: Running, Role: LEADER
I20260809 04:48:34.982821 1187 consensus_queue.cc:237] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6621dcc3ce94c0cb8e5ce9b7ad59c8e" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 39553 } } peers { permanent_uuid: "b9537463e92740d096661779d87c8baa" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 38657 } } peers { permanent_uuid: "3438d9c22fe24859bbbad205af980f1f" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 33095 } }
I20260809 04:48:34.986819 647 catalog_manager.cc:5495] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa reported cstate change: term changed from 0 to 1, leader changed from <none> to b9537463e92740d096661779d87c8baa (127.12.234.1). New cstate: current_term: 1 leader_uuid: "b9537463e92740d096661779d87c8baa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6621dcc3ce94c0cb8e5ce9b7ad59c8e" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9537463e92740d096661779d87c8baa" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 38657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3438d9c22fe24859bbbad205af980f1f" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 33095 } health_report { overall_health: UNKNOWN } } }
I20260809 04:48:35.014189 1177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:35.009488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53856 (local address 127.12.234.1:38657)
0809 04:48:35.009711 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:35.009717 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:35.009735 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:48:35.009886 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:35.009889 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:35.009968 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:35.010080 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:48:35.010414 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:35.011079 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:35.012361 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:35.012715 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:35.012797 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:35.013200 (+ 403us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:48:35.013634 (+ 434us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:48:35.013747 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:35.013890 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 04:48:35.013948 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:48:35.021173 961 raft_consensus.cc:1261] T e30ff1c1b0934d28af6e6532b5ea3cff P 3438d9c22fe24859bbbad205af980f1f [term 1 FOLLOWER]: Refusing update from remote peer b9537463e92740d096661779d87c8baa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:48:35.021296 1112 raft_consensus.cc:1261] T e30ff1c1b0934d28af6e6532b5ea3cff P f6621dcc3ce94c0cb8e5ce9b7ad59c8e [term 1 FOLLOWER]: Refusing update from remote peer b9537463e92740d096661779d87c8baa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:48:35.022056 1191 consensus_queue.cc:1035] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa [LEADER]: Connected to new peer: Peer: permanent_uuid: "3438d9c22fe24859bbbad205af980f1f" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 33095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:48:35.022416 1187 consensus_queue.cc:1035] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6621dcc3ce94c0cb8e5ce9b7ad59c8e" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:48:35.032708 1194 mvcc.cc:205] Tried to move back new op lower bound from 7316483747917680640 to 7316483747773480960. Current Snapshot: MvccSnapshot[applied={T|T < 7316483747917680640}]
I20260809 04:48:35.036687 1196 mvcc.cc:205] Tried to move back new op lower bound from 7316483747917680640 to 7316483747773480960. Current Snapshot: MvccSnapshot[applied={T|T < 7316483747917680640}]
I20260809 04:48:35.044222 647 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:48:35.044468 647 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:48:35.047417 647 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0da19afd0b9b48b0a0646c2b69cb5c3e: Sending DeleteTablet for 3 replicas of tablet e30ff1c1b0934d28af6e6532b5ea3cff
I20260809 04:48:35.048492 1088 tablet_service.cc:1514] Processing DeleteTablet for tablet e30ff1c1b0934d28af6e6532b5ea3cff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:48:35 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:52948
I20260809 04:48:35.048499 787 tablet_service.cc:1514] Processing DeleteTablet for tablet e30ff1c1b0934d28af6e6532b5ea3cff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:48:35 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:53840
I20260809 04:48:35.048641 938 tablet_service.cc:1514] Processing DeleteTablet for tablet e30ff1c1b0934d28af6e6532b5ea3cff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:48:35 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:43500
I20260809 04:48:35.053746 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 715
I20260809 04:48:35.061964 1205 tablet_replica.cc:330] T e30ff1c1b0934d28af6e6532b5ea3cff P 3438d9c22fe24859bbbad205af980f1f: stopping tablet replica
I20260809 04:48:35.062317 1205 raft_consensus.cc:2229] T e30ff1c1b0934d28af6e6532b5ea3cff P 3438d9c22fe24859bbbad205af980f1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:48:35.062757 1205 raft_consensus.cc:2258] T e30ff1c1b0934d28af6e6532b5ea3cff P 3438d9c22fe24859bbbad205af980f1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:48:35.063670 1203 tablet_replica.cc:330] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa: stopping tablet replica
I20260809 04:48:35.064007 1203 raft_consensus.cc:2229] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:48:35.065725 1205 ts_tablet_manager.cc:1905] T e30ff1c1b0934d28af6e6532b5ea3cff P 3438d9c22fe24859bbbad205af980f1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:48:35.072263 1205 ts_tablet_manager.cc:1918] T e30ff1c1b0934d28af6e6532b5ea3cff P 3438d9c22fe24859bbbad205af980f1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:48:35.072448 1205 log.cc:1196] T e30ff1c1b0934d28af6e6532b5ea3cff P 3438d9c22fe24859bbbad205af980f1f: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-1/wal/wals/e30ff1c1b0934d28af6e6532b5ea3cff
I20260809 04:48:35.072430 1203 raft_consensus.cc:2258] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:48:35.073444 1205 ts_tablet_manager.cc:1939] T e30ff1c1b0934d28af6e6532b5ea3cff P 3438d9c22fe24859bbbad205af980f1f: Deleting consensus metadata
I20260809 04:48:35.073477 1204 tablet_replica.cc:330] T e30ff1c1b0934d28af6e6532b5ea3cff P f6621dcc3ce94c0cb8e5ce9b7ad59c8e: stopping tablet replica
I20260809 04:48:35.073975 1204 raft_consensus.cc:2229] T e30ff1c1b0934d28af6e6532b5ea3cff P f6621dcc3ce94c0cb8e5ce9b7ad59c8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:48:35.074540 1204 raft_consensus.cc:2258] T e30ff1c1b0934d28af6e6532b5ea3cff P f6621dcc3ce94c0cb8e5ce9b7ad59c8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:48:35.074751 1203 ts_tablet_manager.cc:1905] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:48:35.075421 631 catalog_manager.cc:4841] TS 3438d9c22fe24859bbbad205af980f1f (127.12.234.2:33095): tablet e30ff1c1b0934d28af6e6532b5ea3cff (table test-table [id=22c5730e457f45d0bca3d200596a5ea0]) successfully deleted
I20260809 04:48:35.076574 1204 ts_tablet_manager.cc:1905] T e30ff1c1b0934d28af6e6532b5ea3cff P f6621dcc3ce94c0cb8e5ce9b7ad59c8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:48:35.081089 1203 ts_tablet_manager.cc:1918] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:48:35.081256 1203 log.cc:1196] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-0/wal/wals/e30ff1c1b0934d28af6e6532b5ea3cff
I20260809 04:48:35.081817 1203 ts_tablet_manager.cc:1939] T e30ff1c1b0934d28af6e6532b5ea3cff P b9537463e92740d096661779d87c8baa: Deleting consensus metadata
Aug 09 04:48:35 dist-test-slave-1jjb krb5kdc[595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.1@KRBTEST.COM
I20260809 04:48:35.083288 1204 ts_tablet_manager.cc:1918] T e30ff1c1b0934d28af6e6532b5ea3cff P f6621dcc3ce94c0cb8e5ce9b7ad59c8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:48:35.083415 1204 log.cc:1196] T e30ff1c1b0934d28af6e6532b5ea3cff P f6621dcc3ce94c0cb8e5ce9b7ad59c8e: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786250603326224-13224-0/minicluster-data/ts-2/wal/wals/e30ff1c1b0934d28af6e6532b5ea3cff
I20260809 04:48:35.083417 628 catalog_manager.cc:4841] TS b9537463e92740d096661779d87c8baa (127.12.234.1:38657): tablet e30ff1c1b0934d28af6e6532b5ea3cff (table test-table [id=22c5730e457f45d0bca3d200596a5ea0]) successfully deleted
I20260809 04:48:35.083909 1204 ts_tablet_manager.cc:1939] T e30ff1c1b0934d28af6e6532b5ea3cff P f6621dcc3ce94c0cb8e5ce9b7ad59c8e: Deleting consensus metadata
I20260809 04:48:35.085439 630 catalog_manager.cc:4841] TS f6621dcc3ce94c0cb8e5ce9b7ad59c8e (127.12.234.3:39553): tablet e30ff1c1b0934d28af6e6532b5ea3cff (table test-table [id=22c5730e457f45d0bca3d200596a5ea0]) successfully deleted
I20260809 04:48:35.090903 1177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:35.057546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53866 (local address 127.12.234.1:38657)
0809 04:48:35.058260 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:35.058270 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:35.064252 (+ 5982us) server_negotiation.cc:407] Connection header received
0809 04:48:35.064557 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:35.064569 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:35.064673 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:35.064752 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:35.066846 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:35.068298 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:35.070850 (+ 2552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:35.071746 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:35.071868 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:35.084568 (+ 12700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:35.084579 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:35.084617 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:35.084659 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:35.087517 (+ 2858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:35.088406 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:35.088415 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:35.088419 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:35.088595 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:35.089096 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:35.089102 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:35.089107 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:35.089653 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:35.089931 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:35.090304 (+ 373us) server_negotiation.cc:299] Negotiation successful
0809 04:48:35.090544 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511}
I20260809 04:48:35.092352 853 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:35.203203 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 715
I20260809 04:48:35.225417 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 865
Aug 09 04:48:35 dist-test-slave-1jjb krb5kdc[595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.2@KRBTEST.COM
I20260809 04:48:35.238919 1179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:35.226111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43514 (local address 127.12.234.2:33095)
0809 04:48:35.226289 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:35.226295 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:35.226385 (+ 90us) server_negotiation.cc:407] Connection header received
0809 04:48:35.226587 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:35.226591 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:35.226649 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:35.226739 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:35.227938 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:35.228734 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:35.230003 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:35.230393 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:35.230510 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:35.234275 (+ 3765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:35.234285 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:35.234298 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:35.234351 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:35.236724 (+ 2373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:35.237321 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:35.237328 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:35.237343 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:35.237454 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:35.237741 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:35.237744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:35.237746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:35.238063 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:35.238297 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:35.238532 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 04:48:35.238677 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 04:48:35.239616 1004 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:35.320678 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 865
I20260809 04:48:35.339917 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 1014
Aug 09 04:48:35 dist-test-slave-1jjb krb5kdc[595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.3@KRBTEST.COM
I20260809 04:48:35.353375 1175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:35.340715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52962 (local address 127.12.234.3:39553)
0809 04:48:35.340864 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:35.340868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:35.340883 (+ 15us) server_negotiation.cc:407] Connection header received
0809 04:48:35.340936 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:35.340939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:35.340993 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:35.341066 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:35.342296 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:35.342996 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:35.344225 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:35.344575 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:35.344661 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:35.348515 (+ 3854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:35.348529 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:35.348546 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:35.348583 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:35.351055 (+ 2472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:35.351611 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:35.351617 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:35.351619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:35.351735 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:35.352024 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:35.352027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:35.352028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:35.352593 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:35.352790 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:35.353004 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 04:48:35.353142 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 04:48:35.353999 1155 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:35.434772 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 1014
I20260809 04:48:35.453151 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 610
I20260809 04:48:35.454160 681 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:35.558336 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 610
2026-08-09T04:48:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4086 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T04:48:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:48:35Z Disabled control of system clock
I20260809 04:48:35.641258 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:46699
--webserver_interface=127.12.234.62
--webserver_port=0
--builtin_ntp_servers=127.12.234.20:35115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:46699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 04:48:35.953251 1216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:35.953549 1216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:35.953636 1216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:35.960393 1216 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:48:35.960477 1216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:35.960531 1216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:35.960588 1216 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:48:35.960640 1216 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:48:35.968786 1216 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:35115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:46699
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:46699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:35.969938 1216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:35.971730 1216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:35.983307 1222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:35.983392 1221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:35.983364 1224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:35.984288 1216 server_base.cc:1034] running on GCE node
I20260809 04:48:35.985086 1216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:35.986465 1216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:35.987682 1216 hybrid_clock.cc:648] HybridClock initialized: now 1786250915987642 us; error 51 us; skew 500 ppm
I20260809 04:48:35.988008 1216 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:48:35.990203 1216 webserver.cc:460] Webserver started at http://127.12.234.62:35577/ using document root <none> and password file <none>
I20260809 04:48:35.990842 1216 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:35.990968 1216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:35.991279 1216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:35.994010 1216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "92c5c36d528941dc85656d5133f38608"
format_stamp: "Formatted at 2026-08-09 04:48:35 on dist-test-slave-1jjb"
server_key: "e273a85596f0db58f6286f16d3ab52f7"
server_key_iv: "b8855b2885d2b50813bbc87fa57c03c0"
server_key_version: "encryptionkey@0"
I20260809 04:48:35.994808 1216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "92c5c36d528941dc85656d5133f38608"
format_stamp: "Formatted at 2026-08-09 04:48:35 on dist-test-slave-1jjb"
server_key: "e273a85596f0db58f6286f16d3ab52f7"
server_key_iv: "b8855b2885d2b50813bbc87fa57c03c0"
server_key_version: "encryptionkey@0"
I20260809 04:48:35.999306 1216 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260809 04:48:36.002449 1233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:36.003686 1216 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260809 04:48:36.003880 1216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "92c5c36d528941dc85656d5133f38608"
format_stamp: "Formatted at 2026-08-09 04:48:35 on dist-test-slave-1jjb"
server_key: "e273a85596f0db58f6286f16d3ab52f7"
server_key_iv: "b8855b2885d2b50813bbc87fa57c03c0"
server_key_version: "encryptionkey@0"
I20260809 04:48:36.004124 1216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:36.028316 1216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:36.029083 1216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:36.029371 1216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:36.046728 1216 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:46699
I20260809 04:48:36.046746 1291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:46699 every 8 connection(s)
I20260809 04:48:36.048003 1216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:48:36.051518 1292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:36.052543 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 1216
I20260809 04:48:36.052722 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:48:36.053157 13224 external_mini_cluster.cc:1428] Setting key c859827fbcdaf172dc02453cf98178dd
I20260809 04:48:36.062611 1292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92c5c36d528941dc85656d5133f38608: Bootstrap starting.
I20260809 04:48:36.065631 1296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:36.054941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46382 (local address 127.12.234.62:46699)
0809 04:48:36.055838 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:36.055847 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:36.055882 (+ 35us) server_negotiation.cc:407] Connection header received
0809 04:48:36.057308 (+ 1426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:36.057328 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:36.057690 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:36.058085 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:36.058596 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:36.059763 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:36.061036 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:36.061581 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:36.061776 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:36.062926 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:36.062935 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:36.062952 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:36.063574 (+ 622us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:36.063660 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:36.063685 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:36.063811 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:36.063988 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 04:48:36.064373 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":325,"threads_started":1}
I20260809 04:48:36.066438 1292 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92c5c36d528941dc85656d5133f38608: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:36.067441 1292 log.cc:824] T 00000000000000000000000000000000 P 92c5c36d528941dc85656d5133f38608: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:36.069975 1292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92c5c36d528941dc85656d5133f38608: No bootstrap required, opened a new log
I20260809 04:48:36.076529 1292 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92c5c36d528941dc85656d5133f38608 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92c5c36d528941dc85656d5133f38608" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46699 } }
I20260809 04:48:36.076834 1292 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92c5c36d528941dc85656d5133f38608 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:36.076951 1292 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92c5c36d528941dc85656d5133f38608 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92c5c36d528941dc85656d5133f38608, State: Initialized, Role: FOLLOWER
I20260809 04:48:36.077528 1292 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92c5c36d528941dc85656d5133f38608 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92c5c36d528941dc85656d5133f38608" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46699 } }
I20260809 04:48:36.077724 1292 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92c5c36d528941dc85656d5133f38608 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:48:36.077811 1292 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92c5c36d528941dc85656d5133f38608 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:48:36.077920 1292 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92c5c36d528941dc85656d5133f38608 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:36.079895 1292 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92c5c36d528941dc85656d5133f38608 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92c5c36d528941dc85656d5133f38608" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46699 } }
I20260809 04:48:36.080408 1292 leader_election.cc:302] T 00000000000000000000000000000000 P 92c5c36d528941dc85656d5133f38608 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92c5c36d528941dc85656d5133f38608; no voters:
I20260809 04:48:36.081001 1292 leader_election.cc:288] T 00000000000000000000000000000000 P 92c5c36d528941dc85656d5133f38608 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:48:36.081166 1298 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92c5c36d528941dc85656d5133f38608 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:48:36.081895 1298 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92c5c36d528941dc85656d5133f38608 [term 1 LEADER]: Becoming Leader. State: Replica: 92c5c36d528941dc85656d5133f38608, State: Running, Role: LEADER
I20260809 04:48:36.082415 1298 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92c5c36d528941dc85656d5133f38608 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92c5c36d528941dc85656d5133f38608" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46699 } }
I20260809 04:48:36.082890 1292 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92c5c36d528941dc85656d5133f38608 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:48:36.085057 1299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92c5c36d528941dc85656d5133f38608 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92c5c36d528941dc85656d5133f38608" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92c5c36d528941dc85656d5133f38608" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46699 } } }
I20260809 04:48:36.085079 1300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92c5c36d528941dc85656d5133f38608 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92c5c36d528941dc85656d5133f38608. Latest consensus state: current_term: 1 leader_uuid: "92c5c36d528941dc85656d5133f38608" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92c5c36d528941dc85656d5133f38608" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46699 } } }
I20260809 04:48:36.085506 1300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92c5c36d528941dc85656d5133f38608 [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:36.087437 1299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92c5c36d528941dc85656d5133f38608 [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:36.088224 1304 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:48:36.093029 1304 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:48:36.100337 1304 catalog_manager.cc:1347] Generated new cluster ID: 13f9406e78c04a7bbab764c172dcc356
I20260809 04:48:36.100430 1304 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:48:36.136976 1304 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:48:36.138159 1304 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:48:36.158546 1304 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92c5c36d528941dc85656d5133f38608: Generated new TSK 0
I20260809 04:48:36.159273 1304 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:48:36.167030 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:0
--local_ip_for_outbound_sockets=127.12.234.1
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:46699
--builtin_ntp_servers=127.12.234.20:35115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 04:48:36.483770 1317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:36.484004 1317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:36.484154 1317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:36.490949 1317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:36.491022 1317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:36.491187 1317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:48:36.499010 1317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:35115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:46699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:36.500123 1317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:36.501652 1317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:36.513787 1327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:36.513918 1323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:36.513877 1324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:36.514722 1317 server_base.cc:1034] running on GCE node
I20260809 04:48:36.515332 1317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:36.516215 1317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:36.517434 1317 hybrid_clock.cc:648] HybridClock initialized: now 1786250916517388 us; error 56 us; skew 500 ppm
I20260809 04:48:36.517717 1317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:48:36.519788 1317 webserver.cc:460] Webserver started at http://127.12.234.1:38041/ using document root <none> and password file <none>
I20260809 04:48:36.520466 1317 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:36.520588 1317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:36.520900 1317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:36.523492 1317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-0/data/instance:
uuid: "d935fcbab53b48d1a99e670e2d968816"
format_stamp: "Formatted at 2026-08-09 04:48:36 on dist-test-slave-1jjb"
server_key: "629e5540e08e18937e57dd05b825c042"
server_key_iv: "a834677f22c330b872b650c2a349aa62"
server_key_version: "encryptionkey@0"
I20260809 04:48:36.524245 1317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance:
uuid: "d935fcbab53b48d1a99e670e2d968816"
format_stamp: "Formatted at 2026-08-09 04:48:36 on dist-test-slave-1jjb"
server_key: "629e5540e08e18937e57dd05b825c042"
server_key_iv: "a834677f22c330b872b650c2a349aa62"
server_key_version: "encryptionkey@0"
I20260809 04:48:36.528636 1317 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260809 04:48:36.531733 1333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:36.533003 1317 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:48:36.533157 1317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "d935fcbab53b48d1a99e670e2d968816"
format_stamp: "Formatted at 2026-08-09 04:48:36 on dist-test-slave-1jjb"
server_key: "629e5540e08e18937e57dd05b825c042"
server_key_iv: "a834677f22c330b872b650c2a349aa62"
server_key_version: "encryptionkey@0"
I20260809 04:48:36.533340 1317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:36.561985 1317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:36.562646 1317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:36.562951 1317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:36.563882 1317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:36.565356 1317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:48:36.565479 1317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:36.565591 1317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:36.565672 1317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:36.602392 1317 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:38869
I20260809 04:48:36.602514 1454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:38869 every 8 connection(s)
I20260809 04:48:36.603554 1317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:48:36.607332 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 1317
I20260809 04:48:36.607510 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance
I20260809 04:48:36.608052 13224 external_mini_cluster.cc:1428] Setting key 48b47f6acaa432b9547df72f920fea68
I20260809 04:48:36.611328 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:0
--local_ip_for_outbound_sockets=127.12.234.2
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:46699
--builtin_ntp_servers=127.12.234.20:35115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 04:48:36.616531 1457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:36.606249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:56207 (local address 127.12.234.62:46699)
0809 04:48:36.606698 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:36.606704 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:36.608119 (+ 1415us) server_negotiation.cc:407] Connection header received
0809 04:48:36.610552 (+ 2433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:36.610557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:36.610636 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:36.610737 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:36.611785 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:36.612513 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:36.613952 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:36.614301 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:36.614426 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:36.615592 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:36.615597 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:36.615599 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:36.615614 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:36.615683 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:36.615705 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:36.615831 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:36.616105 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 04:48:36.616196 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":210,"threads_started":1}
I20260809 04:48:36.617383 1458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:36.606761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46699 (local address 127.12.234.1:56207)
0809 04:48:36.607965 (+ 1204us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:36.608002 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:36.610333 (+ 2331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:36.610939 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:36.610946 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:36.611020 (+ 74us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:48:36.611638 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:36.611656 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:36.612685 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:36.612690 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:36.613801 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:36.613813 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:36.614501 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:36.614507 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:36.614702 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:36.615318 (+ 616us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:48:36.615320 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:48:36.615354 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:48:36.615420 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:48:36.615874 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:48:36.615883 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:48:36.615888 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 04:48:36.616058 (+ 170us) client_negotiation.cc:240] Negotiation successful
0809 04:48:36.616417 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":337,"threads_started":1}
I20260809 04:48:36.618752 1455 heartbeater.cc:347] Connected to a master server at 127.12.234.62:46699
I20260809 04:48:36.619052 1455 heartbeater.cc:464] Registering TS with master...
I20260809 04:48:36.619701 1455 heartbeater.cc:511] Master 127.12.234.62:46699 requested a full tablet report, sending...
I20260809 04:48:36.621438 1252 ts_manager.cc:194] Registered new tserver with Master: d935fcbab53b48d1a99e670e2d968816 (127.12.234.1:38869)
I20260809 04:48:36.622752 1252 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.234.1:56207
W20260809 04:48:36.920696 1459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:36.920914 1459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:36.921046 1459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:36.927151 1459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:36.927224 1459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:36.927305 1459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:48:36.935065 1459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:35115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:46699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:36.936142 1459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:36.937729 1459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:36.949965 1464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:36.950053 1469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:36.950131 1467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:36.951064 1459 server_base.cc:1034] running on GCE node
I20260809 04:48:36.951567 1459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:36.952430 1459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:36.953639 1459 hybrid_clock.cc:648] HybridClock initialized: now 1786250916953596 us; error 39 us; skew 500 ppm
I20260809 04:48:36.953929 1459 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:48:36.955976 1459 webserver.cc:460] Webserver started at http://127.12.234.2:32965/ using document root <none> and password file <none>
I20260809 04:48:36.956591 1459 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:36.956708 1459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:36.956979 1459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:36.959434 1459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-1/data/instance:
uuid: "7a81f58d5fa74d2a80ca3255f025b907"
format_stamp: "Formatted at 2026-08-09 04:48:36 on dist-test-slave-1jjb"
server_key: "02d19c60fc1a9f0a3fdd8413b5b71e35"
server_key_iv: "49ccea9ef8aa80c4b562a7ef7b81c815"
server_key_version: "encryptionkey@0"
I20260809 04:48:36.960184 1459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance:
uuid: "7a81f58d5fa74d2a80ca3255f025b907"
format_stamp: "Formatted at 2026-08-09 04:48:36 on dist-test-slave-1jjb"
server_key: "02d19c60fc1a9f0a3fdd8413b5b71e35"
server_key_iv: "49ccea9ef8aa80c4b562a7ef7b81c815"
server_key_version: "encryptionkey@0"
I20260809 04:48:36.964548 1459 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 04:48:36.967928 1475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:36.969286 1459 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:48:36.969477 1459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "7a81f58d5fa74d2a80ca3255f025b907"
format_stamp: "Formatted at 2026-08-09 04:48:36 on dist-test-slave-1jjb"
server_key: "02d19c60fc1a9f0a3fdd8413b5b71e35"
server_key_iv: "49ccea9ef8aa80c4b562a7ef7b81c815"
server_key_version: "encryptionkey@0"
I20260809 04:48:36.969686 1459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:36.982435 1459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:36.983156 1459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:36.983451 1459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:36.984443 1459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:36.985695 1459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:48:36.985807 1459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:36.985924 1459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:36.986013 1459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:37.016955 1459 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:45477
I20260809 04:48:37.017031 1596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:45477 every 8 connection(s)
I20260809 04:48:37.018175 1459 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:48:37.024407 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 1459
I20260809 04:48:37.024595 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance
I20260809 04:48:37.025101 13224 external_mini_cluster.cc:1428] Setting key 28fbb64ad630b52015f7ae399f9d341f
I20260809 04:48:37.027859 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:0
--local_ip_for_outbound_sockets=127.12.234.3
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:46699
--builtin_ntp_servers=127.12.234.20:35115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 04:48:37.030138 1457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:37.020527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:54051 (local address 127.12.234.62:46699)
0809 04:48:37.020737 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:37.020743 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:37.021918 (+ 1175us) server_negotiation.cc:407] Connection header received
0809 04:48:37.023429 (+ 1511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:37.023433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:37.023495 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:37.023574 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:37.024822 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.025526 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.027272 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.027612 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.027699 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:37.029384 (+ 1685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:37.029390 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:37.029393 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:37.029415 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:37.029452 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:37.029466 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:37.029550 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:37.029816 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 04:48:37.029898 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 04:48:37.031328 1599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:37.020996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46699 (local address 127.12.234.2:54051)
0809 04:48:37.021771 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:37.021802 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:37.023213 (+ 1411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:37.023745 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:37.023752 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:37.023821 (+ 69us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:48:37.024624 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:37.024653 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.025693 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.025696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:37.027136 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:37.027151 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.027815 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.027824 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:37.028141 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:37.028997 (+ 856us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:48:37.029003 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:48:37.029074 (+ 71us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:48:37.029169 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:48:37.029574 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:48:37.029588 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:48:37.029599 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 04:48:37.029781 (+ 182us) client_negotiation.cc:240] Negotiation successful
0809 04:48:37.030159 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":252,"threads_started":1}
I20260809 04:48:37.032666 1597 heartbeater.cc:347] Connected to a master server at 127.12.234.62:46699
I20260809 04:48:37.032974 1597 heartbeater.cc:464] Registering TS with master...
I20260809 04:48:37.033726 1597 heartbeater.cc:511] Master 127.12.234.62:46699 requested a full tablet report, sending...
I20260809 04:48:37.035051 1252 ts_manager.cc:194] Registered new tserver with Master: 7a81f58d5fa74d2a80ca3255f025b907 (127.12.234.2:45477)
I20260809 04:48:37.035876 1252 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.234.2:54051
W20260809 04:48:37.347934 1600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:37.348238 1600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:37.348443 1600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:37.355072 1600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:37.355149 1600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:37.355341 1600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:48:37.363119 1600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:35115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:46699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:37.364089 1600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:37.365533 1600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:37.377157 1610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:37.377223 1606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:37.377193 1608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:37.377830 1600 server_base.cc:1034] running on GCE node
I20260809 04:48:37.378464 1600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:37.379426 1600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:37.380722 1600 hybrid_clock.cc:648] HybridClock initialized: now 1786250917380680 us; error 27 us; skew 500 ppm
I20260809 04:48:37.381059 1600 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:48:37.383397 1600 webserver.cc:460] Webserver started at http://127.12.234.3:38627/ using document root <none> and password file <none>
I20260809 04:48:37.384159 1600 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:37.384279 1600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:37.384617 1600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:37.387349 1600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-2/data/instance:
uuid: "dad70e79d7074156a341e14290110eb0"
format_stamp: "Formatted at 2026-08-09 04:48:37 on dist-test-slave-1jjb"
server_key: "56381e2279935c8cdcd7fe5fafc1ddee"
server_key_iv: "6794fca1e0275a17d61c43c4b5aaa67f"
server_key_version: "encryptionkey@0"
I20260809 04:48:37.388175 1600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance:
uuid: "dad70e79d7074156a341e14290110eb0"
format_stamp: "Formatted at 2026-08-09 04:48:37 on dist-test-slave-1jjb"
server_key: "56381e2279935c8cdcd7fe5fafc1ddee"
server_key_iv: "6794fca1e0275a17d61c43c4b5aaa67f"
server_key_version: "encryptionkey@0"
I20260809 04:48:37.393443 1600 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 04:48:37.396468 1616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:37.397778 1600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 04:48:37.397969 1600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "dad70e79d7074156a341e14290110eb0"
format_stamp: "Formatted at 2026-08-09 04:48:37 on dist-test-slave-1jjb"
server_key: "56381e2279935c8cdcd7fe5fafc1ddee"
server_key_iv: "6794fca1e0275a17d61c43c4b5aaa67f"
server_key_version: "encryptionkey@0"
I20260809 04:48:37.398185 1600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:37.413551 1600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:37.414276 1600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:37.414557 1600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:37.415447 1600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:37.416831 1600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:48:37.416945 1600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:37.417078 1600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:37.417167 1600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:37.449878 1600 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:34331
I20260809 04:48:37.449916 1735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:34331 every 8 connection(s)
I20260809 04:48:37.451181 1600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:48:37.459399 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 1600
I20260809 04:48:37.459605 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance
I20260809 04:48:37.460064 13224 external_mini_cluster.cc:1428] Setting key 7c12340853b976a6f6fdd47585ebf7c4
I20260809 04:48:37.463347 1457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:37.453617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:50433 (local address 127.12.234.62:46699)
0809 04:48:37.453834 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:37.453841 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:37.455106 (+ 1265us) server_negotiation.cc:407] Connection header received
0809 04:48:37.456838 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:37.456842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:37.456908 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:37.456992 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:37.457895 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.458656 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.460579 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.461008 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.461160 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:37.462483 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:37.462488 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:37.462491 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:37.462515 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:37.462547 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:37.462583 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:37.462671 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:37.463048 (+ 377us) server_negotiation.cc:299] Negotiation successful
0809 04:48:37.463122 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 04:48:37.464259 1739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:37.454082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46699 (local address 127.12.234.3:50433)
0809 04:48:37.454957 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:37.454990 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:37.456603 (+ 1613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:37.457177 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:37.457184 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:37.457259 (+ 75us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:48:37.457755 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:37.457770 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.458809 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.458813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:37.460438 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:37.460451 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.461240 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.461249 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:37.461484 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:37.462152 (+ 668us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:48:37.462156 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:48:37.462204 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:48:37.462294 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:48:37.462859 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:48:37.462869 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:48:37.462875 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 04:48:37.463010 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 04:48:37.463255 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":294,"threads_started":1}
I20260809 04:48:37.465508 1736 heartbeater.cc:347] Connected to a master server at 127.12.234.62:46699
I20260809 04:48:37.465783 1736 heartbeater.cc:464] Registering TS with master...
I20260809 04:48:37.466475 1736 heartbeater.cc:511] Master 127.12.234.62:46699 requested a full tablet report, sending...
I20260809 04:48:37.467684 1252 ts_manager.cc:194] Registered new tserver with Master: dad70e79d7074156a341e14290110eb0 (127.12.234.3:34331)
I20260809 04:48:37.468508 1252 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.234.3:50433
I20260809 04:48:37.475343 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:48:37.487058 1457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:37.480758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46394 (local address 127.12.234.62:46699)
0809 04:48:37.480919 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:37.480923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:37.481389 (+ 466us) server_negotiation.cc:407] Connection header received
0809 04:48:37.481591 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:37.481596 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:37.481674 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:37.481775 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:37.482287 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.483121 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.484638 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.485058 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.485189 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:37.486364 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:37.486370 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:37.486374 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:37.486397 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:37.486425 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:37.486436 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:37.486575 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:37.486753 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 04:48:37.486803 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 04:48:37.490876 1252 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46394:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:48:37.497349 1252 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:48:37.520526 1749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:37.508661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:38869 (local address 127.0.0.1:44282)
0809 04:48:37.510955 (+ 2294us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:37.510997 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:37.511170 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:37.513183 (+ 2013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:37.513189 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:37.513216 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:37.513642 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:37.513661 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.515296 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.515302 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:37.517900 (+ 2598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:37.517915 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.519601 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.519607 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:37.519784 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:37.519828 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 04:48:37.519914 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 04:48:37.520168 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2065,"mutex_wait_us":125,"thread_start_us":1941,"threads_started":1}
I20260809 04:48:37.520620 1750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:37.508887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44282 (local address 127.12.234.1:38869)
0809 04:48:37.512541 (+ 3654us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:37.512549 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:37.512577 (+ 28us) server_negotiation.cc:407] Connection header received
0809 04:48:37.512670 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:37.512680 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:37.512950 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:37.513177 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:37.513847 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.515169 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.518108 (+ 2939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.519386 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.519553 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:37.519788 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:37.520128 (+ 340us) server_negotiation.cc:299] Negotiation successful
0809 04:48:37.520286 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3456,"thread_start_us":3326,"threads_started":1}
I20260809 04:48:37.521387 1753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:37.510170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55068 (local address 127.12.234.2:45477)
0809 04:48:37.511818 (+ 1648us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:37.511826 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:37.513364 (+ 1538us) server_negotiation.cc:407] Connection header received
0809 04:48:37.513468 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:37.513474 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:37.513676 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:37.513856 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:37.514653 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.515786 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.518035 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.518902 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.519060 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:37.519332 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:37.520092 (+ 760us) server_negotiation.cc:299] Negotiation successful
0809 04:48:37.521022 (+ 930us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1468,"thread_start_us":1350,"threads_started":1}
I20260809 04:48:37.521768 1755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:37.511896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:34331 (local address 127.0.0.1:46378)
0809 04:48:37.513829 (+ 1933us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:37.513843 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:37.513987 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:37.514671 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:37.514675 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:37.514713 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:37.514983 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:37.514990 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.516801 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.516806 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:37.518854 (+ 2048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:37.518869 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.520993 (+ 2124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.521000 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:37.521177 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:37.521209 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 04:48:37.521290 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 04:48:37.521416 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1819,"mutex_wait_us":21,"thread_start_us":1070,"threads_started":1}
I20260809 04:48:37.522223 1751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:37.509405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:45477 (local address 127.0.0.1:55068)
0809 04:48:37.513105 (+ 3700us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:37.513133 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:37.513240 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:37.514118 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:37.514122 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:37.514141 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:37.514496 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:37.514509 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.515907 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.515915 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:37.517846 (+ 1931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:37.517858 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.519464 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.519470 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:37.519691 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:37.519754 (+ 63us) client_negotiation.cc:769] Sending connection context
0809 04:48:37.519887 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 04:48:37.521904 (+ 2017us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3524,"mutex_wait_us":46,"thread_start_us":2543,"threads_started":1}
I20260809 04:48:37.522545 1754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:37.511612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46378 (local address 127.12.234.3:34331)
0809 04:48:37.512774 (+ 1162us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:37.512781 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:37.513953 (+ 1172us) server_negotiation.cc:407] Connection header received
0809 04:48:37.514127 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:37.514133 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:37.514327 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:37.514509 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:37.515164 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.516641 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.519048 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.520476 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.521667 (+ 1191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:37.521934 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:37.522025 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 04:48:37.522192 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1004,"thread_start_us":879,"threads_started":1}
I20260809 04:48:37.525305 1666 tablet_service.cc:1467] Processing CreateTablet for tablet 4abad5b73c2c40af9b08f9317dee0b33 (DEFAULT_TABLE table=test-table [id=d9610ad82e014c8784f3d9327ecc0ed8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:48:37.525642 1527 tablet_service.cc:1467] Processing CreateTablet for tablet 4abad5b73c2c40af9b08f9317dee0b33 (DEFAULT_TABLE table=test-table [id=d9610ad82e014c8784f3d9327ecc0ed8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:48:37.524895 1384 tablet_service.cc:1467] Processing CreateTablet for tablet 4abad5b73c2c40af9b08f9317dee0b33 (DEFAULT_TABLE table=test-table [id=d9610ad82e014c8784f3d9327ecc0ed8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:48:37.526579 1666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4abad5b73c2c40af9b08f9317dee0b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:37.526791 1527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4abad5b73c2c40af9b08f9317dee0b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:37.527017 1384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4abad5b73c2c40af9b08f9317dee0b33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:37.537701 1756 tablet_bootstrap.cc:492] T 4abad5b73c2c40af9b08f9317dee0b33 P 7a81f58d5fa74d2a80ca3255f025b907: Bootstrap starting.
I20260809 04:48:37.540684 1757 tablet_bootstrap.cc:492] T 4abad5b73c2c40af9b08f9317dee0b33 P d935fcbab53b48d1a99e670e2d968816: Bootstrap starting.
I20260809 04:48:37.542038 1756 tablet_bootstrap.cc:654] T 4abad5b73c2c40af9b08f9317dee0b33 P 7a81f58d5fa74d2a80ca3255f025b907: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:37.542285 1758 tablet_bootstrap.cc:492] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0: Bootstrap starting.
I20260809 04:48:37.543365 1756 log.cc:824] T 4abad5b73c2c40af9b08f9317dee0b33 P 7a81f58d5fa74d2a80ca3255f025b907: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:37.544150 1757 tablet_bootstrap.cc:654] T 4abad5b73c2c40af9b08f9317dee0b33 P d935fcbab53b48d1a99e670e2d968816: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:37.545197 1757 log.cc:824] T 4abad5b73c2c40af9b08f9317dee0b33 P d935fcbab53b48d1a99e670e2d968816: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:37.545323 1758 tablet_bootstrap.cc:654] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:37.546289 1756 tablet_bootstrap.cc:492] T 4abad5b73c2c40af9b08f9317dee0b33 P 7a81f58d5fa74d2a80ca3255f025b907: No bootstrap required, opened a new log
I20260809 04:48:37.546459 1758 log.cc:824] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:37.546550 1756 ts_tablet_manager.cc:1397] T 4abad5b73c2c40af9b08f9317dee0b33 P 7a81f58d5fa74d2a80ca3255f025b907: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260809 04:48:37.548007 1757 tablet_bootstrap.cc:492] T 4abad5b73c2c40af9b08f9317dee0b33 P d935fcbab53b48d1a99e670e2d968816: No bootstrap required, opened a new log
I20260809 04:48:37.548300 1757 ts_tablet_manager.cc:1397] T 4abad5b73c2c40af9b08f9317dee0b33 P d935fcbab53b48d1a99e670e2d968816: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260809 04:48:37.549220 1758 tablet_bootstrap.cc:492] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0: No bootstrap required, opened a new log
I20260809 04:48:37.549470 1758 ts_tablet_manager.cc:1397] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260809 04:48:37.555357 1758 raft_consensus.cc:348] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d935fcbab53b48d1a99e670e2d968816" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 38869 } } peers { permanent_uuid: "dad70e79d7074156a341e14290110eb0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 34331 } } peers { permanent_uuid: "7a81f58d5fa74d2a80ca3255f025b907" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45477 } }
I20260809 04:48:37.555678 1758 raft_consensus.cc:374] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:37.555794 1758 raft_consensus.cc:729] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dad70e79d7074156a341e14290110eb0, State: Initialized, Role: FOLLOWER
I20260809 04:48:37.556437 1758 consensus_queue.cc:260] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d935fcbab53b48d1a99e670e2d968816" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 38869 } } peers { permanent_uuid: "dad70e79d7074156a341e14290110eb0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 34331 } } peers { permanent_uuid: "7a81f58d5fa74d2a80ca3255f025b907" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45477 } }
I20260809 04:48:37.556597 1756 raft_consensus.cc:348] T 4abad5b73c2c40af9b08f9317dee0b33 P 7a81f58d5fa74d2a80ca3255f025b907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d935fcbab53b48d1a99e670e2d968816" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 38869 } } peers { permanent_uuid: "dad70e79d7074156a341e14290110eb0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 34331 } } peers { permanent_uuid: "7a81f58d5fa74d2a80ca3255f025b907" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45477 } }
I20260809 04:48:37.556999 1756 raft_consensus.cc:374] T 4abad5b73c2c40af9b08f9317dee0b33 P 7a81f58d5fa74d2a80ca3255f025b907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:37.557101 1756 raft_consensus.cc:729] T 4abad5b73c2c40af9b08f9317dee0b33 P 7a81f58d5fa74d2a80ca3255f025b907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a81f58d5fa74d2a80ca3255f025b907, State: Initialized, Role: FOLLOWER
I20260809 04:48:37.557781 1756 consensus_queue.cc:260] T 4abad5b73c2c40af9b08f9317dee0b33 P 7a81f58d5fa74d2a80ca3255f025b907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d935fcbab53b48d1a99e670e2d968816" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 38869 } } peers { permanent_uuid: "dad70e79d7074156a341e14290110eb0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 34331 } } peers { permanent_uuid: "7a81f58d5fa74d2a80ca3255f025b907" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45477 } }
I20260809 04:48:37.558205 1758 ts_tablet_manager.cc:1428] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0: Time spent starting tablet: real 0.009s user 0.003s sys 0.006s
I20260809 04:48:37.557971 1757 raft_consensus.cc:348] T 4abad5b73c2c40af9b08f9317dee0b33 P d935fcbab53b48d1a99e670e2d968816 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d935fcbab53b48d1a99e670e2d968816" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 38869 } } peers { permanent_uuid: "dad70e79d7074156a341e14290110eb0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 34331 } } peers { permanent_uuid: "7a81f58d5fa74d2a80ca3255f025b907" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45477 } }
I20260809 04:48:37.558415 1757 raft_consensus.cc:374] T 4abad5b73c2c40af9b08f9317dee0b33 P d935fcbab53b48d1a99e670e2d968816 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:37.558534 1757 raft_consensus.cc:729] T 4abad5b73c2c40af9b08f9317dee0b33 P d935fcbab53b48d1a99e670e2d968816 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d935fcbab53b48d1a99e670e2d968816, State: Initialized, Role: FOLLOWER
I20260809 04:48:37.558876 1736 heartbeater.cc:503] Master 127.12.234.62:46699 was elected leader, sending a full tablet report...
I20260809 04:48:37.559399 1597 heartbeater.cc:503] Master 127.12.234.62:46699 was elected leader, sending a full tablet report...
I20260809 04:48:37.559238 1757 consensus_queue.cc:260] T 4abad5b73c2c40af9b08f9317dee0b33 P d935fcbab53b48d1a99e670e2d968816 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d935fcbab53b48d1a99e670e2d968816" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 38869 } } peers { permanent_uuid: "dad70e79d7074156a341e14290110eb0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 34331 } } peers { permanent_uuid: "7a81f58d5fa74d2a80ca3255f025b907" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45477 } }
I20260809 04:48:37.560011 1756 ts_tablet_manager.cc:1428] T 4abad5b73c2c40af9b08f9317dee0b33 P 7a81f58d5fa74d2a80ca3255f025b907: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260809 04:48:37.561345 1455 heartbeater.cc:503] Master 127.12.234.62:46699 was elected leader, sending a full tablet report...
I20260809 04:48:37.561952 1757 ts_tablet_manager.cc:1428] T 4abad5b73c2c40af9b08f9317dee0b33 P d935fcbab53b48d1a99e670e2d968816: Time spent starting tablet: real 0.014s user 0.004s sys 0.008s
W20260809 04:48:37.605847 1456 tablet.cc:2369] T 4abad5b73c2c40af9b08f9317dee0b33 P d935fcbab53b48d1a99e670e2d968816: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:48:37.702867 1738 tablet.cc:2369] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:48:37.770342 1598 tablet.cc:2369] T 4abad5b73c2c40af9b08f9317dee0b33 P 7a81f58d5fa74d2a80ca3255f025b907: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:48:37.965739 1762 raft_consensus.cc:482] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:48:37.966032 1762 raft_consensus.cc:504] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d935fcbab53b48d1a99e670e2d968816" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 38869 } } peers { permanent_uuid: "dad70e79d7074156a341e14290110eb0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 34331 } } peers { permanent_uuid: "7a81f58d5fa74d2a80ca3255f025b907" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45477 } }
I20260809 04:48:37.967504 1762 leader_election.cc:288] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d935fcbab53b48d1a99e670e2d968816 (127.12.234.1:38869), 7a81f58d5fa74d2a80ca3255f025b907 (127.12.234.2:45477)
I20260809 04:48:37.975198 1750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:37.967795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:53351 (local address 127.12.234.1:38869)
0809 04:48:37.967979 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:37.967985 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:37.968694 (+ 709us) server_negotiation.cc:407] Connection header received
0809 04:48:37.968929 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:37.968933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:37.969006 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:37.969125 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:37.969718 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.970388 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.972889 (+ 2501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.974121 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.974280 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:37.974351 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:37.974824 (+ 473us) server_negotiation.cc:299] Negotiation successful
0809 04:48:37.974963 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 04:48:37.975199 1753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:37.967936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:35081 (local address 127.12.234.2:45477)
0809 04:48:37.968145 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:37.968154 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:37.968921 (+ 767us) server_negotiation.cc:407] Connection header received
0809 04:48:37.969116 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:37.969119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:37.969178 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:37.969275 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:37.969938 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.970661 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.972876 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.974122 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.974299 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:37.974371 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:37.974824 (+ 453us) server_negotiation.cc:299] Negotiation successful
0809 04:48:37.974951 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 04:48:37.975308 1766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:37.967765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:38869 (local address 127.12.234.3:53351)
0809 04:48:37.968574 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:37.968593 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:37.968770 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:37.969143 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:37.969147 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:37.969171 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:37.969538 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:37.969553 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.970524 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.970532 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:37.972658 (+ 2126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:37.972674 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.974279 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.974287 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:37.974579 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:37.974687 (+ 108us) client_negotiation.cc:769] Sending connection context
0809 04:48:37.974781 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 04:48:37.974933 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"mutex_wait_us":98,"thread_start_us":559,"threads_started":1}
I20260809 04:48:37.975333 1767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:37.967924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:45477 (local address 127.12.234.3:35081)
0809 04:48:37.968810 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:37.968825 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:37.968966 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:37.969295 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:37.969299 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:37.969322 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:37.969757 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:37.969766 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.970806 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.970815 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:37.972699 (+ 1884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:37.972712 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:37.974305 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:37.974310 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:37.974607 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:37.974689 (+ 82us) client_negotiation.cc:769] Sending connection context
0809 04:48:37.974781 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 04:48:37.974933 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"mutex_wait_us":120,"thread_start_us":602,"threads_started":1}
I20260809 04:48:37.976406 1406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4abad5b73c2c40af9b08f9317dee0b33" candidate_uuid: "dad70e79d7074156a341e14290110eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d935fcbab53b48d1a99e670e2d968816" is_pre_election: true
I20260809 04:48:37.976429 1549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4abad5b73c2c40af9b08f9317dee0b33" candidate_uuid: "dad70e79d7074156a341e14290110eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a81f58d5fa74d2a80ca3255f025b907" is_pre_election: true
I20260809 04:48:37.976898 1549 raft_consensus.cc:2454] T 4abad5b73c2c40af9b08f9317dee0b33 P 7a81f58d5fa74d2a80ca3255f025b907 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dad70e79d7074156a341e14290110eb0 in term 0.
I20260809 04:48:37.976909 1406 raft_consensus.cc:2454] T 4abad5b73c2c40af9b08f9317dee0b33 P d935fcbab53b48d1a99e670e2d968816 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dad70e79d7074156a341e14290110eb0 in term 0.
I20260809 04:48:37.977486 1618 leader_election.cc:302] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a81f58d5fa74d2a80ca3255f025b907, dad70e79d7074156a341e14290110eb0; no voters:
I20260809 04:48:37.977837 1762 raft_consensus.cc:2781] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:48:37.977989 1762 raft_consensus.cc:482] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:48:37.978080 1762 raft_consensus.cc:3037] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:37.980331 1762 raft_consensus.cc:504] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d935fcbab53b48d1a99e670e2d968816" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 38869 } } peers { permanent_uuid: "dad70e79d7074156a341e14290110eb0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 34331 } } peers { permanent_uuid: "7a81f58d5fa74d2a80ca3255f025b907" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45477 } }
I20260809 04:48:37.980861 1762 leader_election.cc:288] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0 [CANDIDATE]: Term 1 election: Requested vote from peers d935fcbab53b48d1a99e670e2d968816 (127.12.234.1:38869), 7a81f58d5fa74d2a80ca3255f025b907 (127.12.234.2:45477)
I20260809 04:48:37.981211 1406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4abad5b73c2c40af9b08f9317dee0b33" candidate_uuid: "dad70e79d7074156a341e14290110eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d935fcbab53b48d1a99e670e2d968816"
I20260809 04:48:37.981324 1549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4abad5b73c2c40af9b08f9317dee0b33" candidate_uuid: "dad70e79d7074156a341e14290110eb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a81f58d5fa74d2a80ca3255f025b907"
I20260809 04:48:37.981392 1406 raft_consensus.cc:3037] T 4abad5b73c2c40af9b08f9317dee0b33 P d935fcbab53b48d1a99e670e2d968816 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:37.981525 1549 raft_consensus.cc:3037] T 4abad5b73c2c40af9b08f9317dee0b33 P 7a81f58d5fa74d2a80ca3255f025b907 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:37.983630 1406 raft_consensus.cc:2454] T 4abad5b73c2c40af9b08f9317dee0b33 P d935fcbab53b48d1a99e670e2d968816 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dad70e79d7074156a341e14290110eb0 in term 1.
I20260809 04:48:37.983670 1549 raft_consensus.cc:2454] T 4abad5b73c2c40af9b08f9317dee0b33 P 7a81f58d5fa74d2a80ca3255f025b907 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dad70e79d7074156a341e14290110eb0 in term 1.
I20260809 04:48:37.984135 1620 leader_election.cc:302] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d935fcbab53b48d1a99e670e2d968816, dad70e79d7074156a341e14290110eb0; no voters:
I20260809 04:48:37.984452 1762 raft_consensus.cc:2781] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:48:37.984812 1762 raft_consensus.cc:686] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0 [term 1 LEADER]: Becoming Leader. State: Replica: dad70e79d7074156a341e14290110eb0, State: Running, Role: LEADER
I20260809 04:48:37.985219 1762 consensus_queue.cc:237] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d935fcbab53b48d1a99e670e2d968816" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 38869 } } peers { permanent_uuid: "dad70e79d7074156a341e14290110eb0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 34331 } } peers { permanent_uuid: "7a81f58d5fa74d2a80ca3255f025b907" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45477 } }
I20260809 04:48:37.989310 1251 catalog_manager.cc:5495] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0 reported cstate change: term changed from 0 to 1, leader changed from <none> to dad70e79d7074156a341e14290110eb0 (127.12.234.3). New cstate: current_term: 1 leader_uuid: "dad70e79d7074156a341e14290110eb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d935fcbab53b48d1a99e670e2d968816" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 38869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dad70e79d7074156a341e14290110eb0" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 34331 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a81f58d5fa74d2a80ca3255f025b907" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45477 } health_report { overall_health: UNKNOWN } } }
I20260809 04:48:38.080984 1772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:38.074956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46382 (local address 127.12.234.3:34331)
0809 04:48:38.075433 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:38.075439 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:38.075710 (+ 271us) server_negotiation.cc:407] Connection header received
0809 04:48:38.075857 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:38.075860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:38.075937 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:38.076057 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:48:38.076487 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:38.077146 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:38.078605 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:38.079050 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:38.079144 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:38.079677 (+ 533us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:48:38.080290 (+ 613us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:48:38.080440 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:38.080605 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 04:48:38.080668 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":221,"threads_started":1}
I20260809 04:48:38.088876 1549 raft_consensus.cc:1261] T 4abad5b73c2c40af9b08f9317dee0b33 P 7a81f58d5fa74d2a80ca3255f025b907 [term 1 FOLLOWER]: Refusing update from remote peer dad70e79d7074156a341e14290110eb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:48:38.088874 1406 raft_consensus.cc:1261] T 4abad5b73c2c40af9b08f9317dee0b33 P d935fcbab53b48d1a99e670e2d968816 [term 1 FOLLOWER]: Refusing update from remote peer dad70e79d7074156a341e14290110eb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:48:38.089987 1769 consensus_queue.cc:1035] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d935fcbab53b48d1a99e670e2d968816" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 38869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:48:38.090339 1762 consensus_queue.cc:1035] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a81f58d5fa74d2a80ca3255f025b907" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 45477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:48:38.100884 1774 mvcc.cc:205] Tried to move back new op lower bound from 7316483760481304576 to 7316483760070750208. Current Snapshot: MvccSnapshot[applied={T|T < 7316483760481304576}]
I20260809 04:48:38.115984 1251 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:48:38.116253 1251 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:48:38.119267 1251 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 92c5c36d528941dc85656d5133f38608: Sending DeleteTablet for 3 replicas of tablet 4abad5b73c2c40af9b08f9317dee0b33
I20260809 04:48:38.120301 1384 tablet_service.cc:1514] Processing DeleteTablet for tablet 4abad5b73c2c40af9b08f9317dee0b33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:48:38 UTC) from {username='slave'} at 127.0.0.1:44282
I20260809 04:48:38.120371 1666 tablet_service.cc:1514] Processing DeleteTablet for tablet 4abad5b73c2c40af9b08f9317dee0b33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:48:38 UTC) from {username='slave'} at 127.0.0.1:46378
I20260809 04:48:38.120427 1527 tablet_service.cc:1514] Processing DeleteTablet for tablet 4abad5b73c2c40af9b08f9317dee0b33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:48:38 UTC) from {username='slave'} at 127.0.0.1:55068
I20260809 04:48:38.124567 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 1317
I20260809 04:48:38.148552 1789 tablet_replica.cc:330] T 4abad5b73c2c40af9b08f9317dee0b33 P 7a81f58d5fa74d2a80ca3255f025b907: stopping tablet replica
I20260809 04:48:38.148973 1789 raft_consensus.cc:2229] T 4abad5b73c2c40af9b08f9317dee0b33 P 7a81f58d5fa74d2a80ca3255f025b907 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:48:38.149488 1789 raft_consensus.cc:2258] T 4abad5b73c2c40af9b08f9317dee0b33 P 7a81f58d5fa74d2a80ca3255f025b907 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:48:38.153923 1789 ts_tablet_manager.cc:1905] T 4abad5b73c2c40af9b08f9317dee0b33 P 7a81f58d5fa74d2a80ca3255f025b907: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:48:38.158053 1750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:38.125827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44294 (local address 127.12.234.1:38869)
0809 04:48:38.125996 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:38.126001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:38.145116 (+ 19115us) server_negotiation.cc:407] Connection header received
0809 04:48:38.145558 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:38.145564 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:38.145663 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:38.145759 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:38.146354 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:38.147462 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:38.150971 (+ 3509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:38.151592 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:38.151680 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:38.156479 (+ 4799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:38.156490 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:38.156504 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:38.157192 (+ 688us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:38.157245 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:38.157268 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:38.157351 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:38.157600 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 04:48:38.157700 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 04:48:38.159183 1450 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:38.159312 1787 tablet_replica.cc:330] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0: stopping tablet replica
I20260809 04:48:38.159693 1787 raft_consensus.cc:2229] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:48:38.160288 1787 raft_consensus.cc:2258] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:48:38.162016 1787 ts_tablet_manager.cc:1905] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:48:38.168561 1789 ts_tablet_manager.cc:1918] T 4abad5b73c2c40af9b08f9317dee0b33 P 7a81f58d5fa74d2a80ca3255f025b907: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:48:38.168722 1789 log.cc:1196] T 4abad5b73c2c40af9b08f9317dee0b33 P 7a81f58d5fa74d2a80ca3255f025b907: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-1/wal/wals/4abad5b73c2c40af9b08f9317dee0b33
I20260809 04:48:38.169265 1789 ts_tablet_manager.cc:1939] T 4abad5b73c2c40af9b08f9317dee0b33 P 7a81f58d5fa74d2a80ca3255f025b907: Deleting consensus metadata
I20260809 04:48:38.171051 1235 catalog_manager.cc:4841] TS 7a81f58d5fa74d2a80ca3255f025b907 (127.12.234.2:45477): tablet 4abad5b73c2c40af9b08f9317dee0b33 (table test-table [id=d9610ad82e014c8784f3d9327ecc0ed8]) successfully deleted
I20260809 04:48:38.176412 1787 ts_tablet_manager.cc:1918] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:48:38.176579 1787 log.cc:1196] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786250603326224-13224-0/minicluster-data/ts-2/wal/wals/4abad5b73c2c40af9b08f9317dee0b33
I20260809 04:48:38.177057 1787 ts_tablet_manager.cc:1939] T 4abad5b73c2c40af9b08f9317dee0b33 P dad70e79d7074156a341e14290110eb0: Deleting consensus metadata
I20260809 04:48:38.178601 1237 catalog_manager.cc:4841] TS dad70e79d7074156a341e14290110eb0 (127.12.234.3:34331): tablet 4abad5b73c2c40af9b08f9317dee0b33 (table test-table [id=d9610ad82e014c8784f3d9327ecc0ed8]) successfully deleted
I20260809 04:48:38.265550 1786 tablet_replica.cc:330] T 4abad5b73c2c40af9b08f9317dee0b33 P d935fcbab53b48d1a99e670e2d968816: stopping tablet replica
I20260809 04:48:38.265765 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 1317
W20260809 04:48:38.292356 1237 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.234.1:38869 (error 108)
I20260809 04:48:38.292639 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 1459
I20260809 04:48:38.295269 1794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:38.294148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:38869 (local address 127.0.0.1:44300)
0809 04:48:38.294983 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:38.295111 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:38869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":713,"thread_start_us":571,"threads_started":1}
W20260809 04:48:38.295588 1237 catalog_manager.cc:4584] TS d935fcbab53b48d1a99e670e2d968816 (127.12.234.1:38869): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4abad5b73c2c40af9b08f9317dee0b33: Network error: Client connection negotiation failed: client connection to 127.12.234.1:38869: connect: Connection refused (error 111)
I20260809 04:48:38.300879 1753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:38.293834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55082 (local address 127.12.234.2:45477)
0809 04:48:38.294033 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:38.294039 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:38.294060 (+ 21us) server_negotiation.cc:407] Connection header received
0809 04:48:38.294130 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:38.294135 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:38.294272 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:38.294415 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:38.294960 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:38.296135 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:38.297398 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:38.298017 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:38.298149 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:38.299198 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:38.299210 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:38.299228 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:38.300066 (+ 838us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:38.300159 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:38.300194 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:38.300317 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:38.300392 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 04:48:38.300504 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 04:48:38.301677 1592 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:38.349571 1794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:38.349013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:38869 (local address 127.0.0.1:44306)
0809 04:48:38.349284 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:38.349386 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:38869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260809 04:48:38.350733 1794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:38.350333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:38869 (local address 127.0.0.1:44312)
0809 04:48:38.350487 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:38.350580 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:38869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 04:48:38.384163 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 1459
I20260809 04:48:38.405680 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 1600
I20260809 04:48:38.411339 1794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:38.410887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:38869 (local address 127.0.0.1:44326)
0809 04:48:38.411126 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:38.411196 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:38869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
I20260809 04:48:38.412669 1794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:38.412017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:38869 (local address 127.0.0.1:44336)
0809 04:48:38.412450 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:38.412525 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:38869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 04:48:38.413843 1772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:38.406506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46392 (local address 127.12.234.3:34331)
0809 04:48:38.406671 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:38.406677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:38.406775 (+ 98us) server_negotiation.cc:407] Connection header received
0809 04:48:38.407010 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:38.407014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:38.407142 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:38.407264 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:38.407705 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:38.408670 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:38.410055 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:38.410696 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:38.410832 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:38.412315 (+ 1483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:38.412330 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:38.412351 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:38.413113 (+ 762us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:38.413161 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:38.413180 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:38.413256 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:38.413442 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 04:48:38.413528 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:48:38.414543 1731 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:38.493556 1794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:38.493071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:38869 (local address 127.0.0.1:44342)
0809 04:48:38.493293 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:38.493403 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:38869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 04:48:38.494658 1794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:38.494242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:38869 (local address 127.0.0.1:44354)
0809 04:48:38.494465 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:38.494538 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.1:38869: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260809 04:48:38.526723 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 1600
I20260809 04:48:38.548784 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 1216
I20260809 04:48:38.549856 1286 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:38.648840 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 1216
2026-08-09T04:48:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3090 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:48:38 dist-test-slave-1jjb krb5kdc[1800](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:48:38 dist-test-slave-1jjb krb5kdc[1800](info): set up 2 sockets
Aug 09 04:48:38 dist-test-slave-1jjb krb5kdc[1800](info): commencing operation
krb5kdc: starting...
W20260809 04:48:39.731106 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.002s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:48:39 dist-test-slave-1jjb krb5kdc[1800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:48:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:48:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:48:39.975625 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:44411
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:40221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:44411
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:48:40.287561 1814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:40.287798 1814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:40.287943 1814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:40.295250 1814 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:48:40.295348 1814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:40.295423 1814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:40.295514 1814 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:48:40.295573 1814 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:48:40.305168 1814 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:40221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:44411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:44411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:40.306416 1814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:40.307761 1814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:40.319505 1820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:40.319450 1824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:40.319548 1822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:40.320410 1814 server_base.cc:1034] running on GCE node
I20260809 04:48:40.321188 1814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:40.322487 1814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:40.323716 1814 hybrid_clock.cc:648] HybridClock initialized: now 1786250920323679 us; error 47 us; skew 500 ppm
I20260809 04:48:40.324024 1814 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:48:40 dist-test-slave-1jjb krb5kdc[1800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250920, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:40.332662 1814 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:48:40.334061 1814 webserver.cc:460] Webserver started at http://127.12.234.62:33227/ using document root <none> and password file <none>
I20260809 04:48:40.334683 1814 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:40.334784 1814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:40.335026 1814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:40.337864 1814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "0c2c8e1f1c984cd48e5598baa6d82bfc"
format_stamp: "Formatted at 2026-08-09 04:48:40 on dist-test-slave-1jjb"
server_key: "6065cc9efb5357922889618fce6b1ae4"
server_key_iv: "a4c4a6ce79fa4633726c778d713a4bb0"
server_key_version: "encryptionkey@0"
I20260809 04:48:40.338541 1814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "0c2c8e1f1c984cd48e5598baa6d82bfc"
format_stamp: "Formatted at 2026-08-09 04:48:40 on dist-test-slave-1jjb"
server_key: "6065cc9efb5357922889618fce6b1ae4"
server_key_iv: "a4c4a6ce79fa4633726c778d713a4bb0"
server_key_version: "encryptionkey@0"
I20260809 04:48:40.343318 1814 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 04:48:40.346490 1833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:40.347750 1814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:48:40.347886 1814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "0c2c8e1f1c984cd48e5598baa6d82bfc"
format_stamp: "Formatted at 2026-08-09 04:48:40 on dist-test-slave-1jjb"
server_key: "6065cc9efb5357922889618fce6b1ae4"
server_key_iv: "a4c4a6ce79fa4633726c778d713a4bb0"
server_key_version: "encryptionkey@0"
I20260809 04:48:40.348035 1814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:40.371553 1814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:40.375465 1814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:40.375696 1814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:40.394589 1814 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:44411
I20260809 04:48:40.394693 1890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:44411 every 8 connection(s)
I20260809 04:48:40.395849 1814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:48:40.397655 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 1814
I20260809 04:48:40.397838 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:48:40.398273 13224 external_mini_cluster.cc:1428] Setting key 4a4fe6b4d1797db802a34ba5e44130ce
I20260809 04:48:40.400130 1891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 04:48:40 dist-test-slave-1jjb krb5kdc[1800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:48:40.414361 1891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c2c8e1f1c984cd48e5598baa6d82bfc: Bootstrap starting.
I20260809 04:48:40.418895 1891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c2c8e1f1c984cd48e5598baa6d82bfc: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:40.420256 1891 log.cc:824] T 00000000000000000000000000000000 P 0c2c8e1f1c984cd48e5598baa6d82bfc: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:40.423581 1895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:40.400140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49284 (local address 127.12.234.62:44411)
0809 04:48:40.401008 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:40.401033 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:40.401086 (+ 53us) server_negotiation.cc:407] Connection header received
0809 04:48:40.403400 (+ 2314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:40.403426 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:40.403941 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:40.404414 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:40.405811 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:40.407904 (+ 2093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:40.409224 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:40.410292 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:40.410500 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:40.414603 (+ 4103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:40.414615 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:40.414632 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:40.414678 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:40.418778 (+ 4100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:40.419491 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:40.419500 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:40.419508 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:40.419738 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:40.420168 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:40.420175 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:40.420178 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:40.420936 (+ 758us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:40.421246 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:40.421766 (+ 520us) server_negotiation.cc:299] Negotiation successful
0809 04:48:40.422173 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":349,"threads_started":1}
I20260809 04:48:40.423779 1891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c2c8e1f1c984cd48e5598baa6d82bfc: No bootstrap required, opened a new log
I20260809 04:48:40.432297 1891 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c2c8e1f1c984cd48e5598baa6d82bfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c2c8e1f1c984cd48e5598baa6d82bfc" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44411 } }
I20260809 04:48:40.432608 1891 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c2c8e1f1c984cd48e5598baa6d82bfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:40.432677 1891 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c2c8e1f1c984cd48e5598baa6d82bfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c2c8e1f1c984cd48e5598baa6d82bfc, State: Initialized, Role: FOLLOWER
I20260809 04:48:40.433215 1891 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c2c8e1f1c984cd48e5598baa6d82bfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c2c8e1f1c984cd48e5598baa6d82bfc" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44411 } }
I20260809 04:48:40.433424 1891 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c2c8e1f1c984cd48e5598baa6d82bfc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:48:40.433518 1891 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c2c8e1f1c984cd48e5598baa6d82bfc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:48:40.433657 1891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c2c8e1f1c984cd48e5598baa6d82bfc [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:40.435694 1891 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c2c8e1f1c984cd48e5598baa6d82bfc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c2c8e1f1c984cd48e5598baa6d82bfc" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44411 } }
I20260809 04:48:40.436157 1891 leader_election.cc:302] T 00000000000000000000000000000000 P 0c2c8e1f1c984cd48e5598baa6d82bfc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c2c8e1f1c984cd48e5598baa6d82bfc; no voters:
I20260809 04:48:40.436663 1891 leader_election.cc:288] T 00000000000000000000000000000000 P 0c2c8e1f1c984cd48e5598baa6d82bfc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:48:40.436909 1897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c2c8e1f1c984cd48e5598baa6d82bfc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:48:40.437923 1897 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c2c8e1f1c984cd48e5598baa6d82bfc [term 1 LEADER]: Becoming Leader. State: Replica: 0c2c8e1f1c984cd48e5598baa6d82bfc, State: Running, Role: LEADER
I20260809 04:48:40.438447 1897 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c2c8e1f1c984cd48e5598baa6d82bfc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c2c8e1f1c984cd48e5598baa6d82bfc" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44411 } }
I20260809 04:48:40.438793 1891 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c2c8e1f1c984cd48e5598baa6d82bfc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:48:40.441252 1899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c2c8e1f1c984cd48e5598baa6d82bfc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c2c8e1f1c984cd48e5598baa6d82bfc. Latest consensus state: current_term: 1 leader_uuid: "0c2c8e1f1c984cd48e5598baa6d82bfc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c2c8e1f1c984cd48e5598baa6d82bfc" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44411 } } }
I20260809 04:48:40.441681 1899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c2c8e1f1c984cd48e5598baa6d82bfc [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:40.441540 1898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c2c8e1f1c984cd48e5598baa6d82bfc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c2c8e1f1c984cd48e5598baa6d82bfc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c2c8e1f1c984cd48e5598baa6d82bfc" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 44411 } } }
I20260809 04:48:40.442054 1898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c2c8e1f1c984cd48e5598baa6d82bfc [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:40.444759 1906 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:48:40.449932 1906 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:48:40.458731 1906 catalog_manager.cc:1347] Generated new cluster ID: 6ea7945a075c49d99a06ac523b232af5
I20260809 04:48:40.458851 1906 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:48:40.479523 1906 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:48:40.480937 1906 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:48:40.490597 1906 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c2c8e1f1c984cd48e5598baa6d82bfc: Generated new TSK 0
I20260809 04:48:40.491369 1906 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.234.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
Entry for principal kudu/127.12.234.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab.
I20260809 04:48:40.573868 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:0
--local_ip_for_outbound_sockets=127.12.234.1
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44411
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:40221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:48:41.008427 1919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:41.008697 1919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:41.008886 1919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:41.016009 1919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:41.016162 1919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:41.016356 1919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:48:41.025779 1919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:40221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.1.keytab
--principal=kudu/127.12.234.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44411
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:41.027062 1919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:41.028388 1919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:41.041196 1925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:41.041343 1927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:41.041236 1924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:41.041980 1919 server_base.cc:1034] running on GCE node
I20260809 04:48:41.042544 1919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:41.043395 1919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:41.044626 1919 hybrid_clock.cc:648] HybridClock initialized: now 1786250921044574 us; error 42 us; skew 500 ppm
I20260809 04:48:41.044948 1919 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:48:41 dist-test-slave-1jjb krb5kdc[1800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250921, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:41.048117 1919 init.cc:377] Logged in from keytab as kudu/127.12.234.1@KRBTEST.COM (short username kudu)
I20260809 04:48:41.049497 1919 webserver.cc:460] Webserver started at http://127.12.234.1:40181/ using document root <none> and password file <none>
I20260809 04:48:41.050165 1919 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:41.050323 1919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:41.050635 1919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:41.053524 1919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-0/data/instance:
uuid: "dcf3c47f2a15418fad08b8ea9d05775c"
format_stamp: "Formatted at 2026-08-09 04:48:41 on dist-test-slave-1jjb"
server_key: "7bdfc0f989796f0d4ee824793ece9d80"
server_key_iv: "2bd0502f9a0d469c73679c438f56b957"
server_key_version: "encryptionkey@0"
I20260809 04:48:41.054289 1919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance:
uuid: "dcf3c47f2a15418fad08b8ea9d05775c"
format_stamp: "Formatted at 2026-08-09 04:48:41 on dist-test-slave-1jjb"
server_key: "7bdfc0f989796f0d4ee824793ece9d80"
server_key_iv: "2bd0502f9a0d469c73679c438f56b957"
server_key_version: "encryptionkey@0"
I20260809 04:48:41.059099 1919 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 04:48:41.062318 1935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:41.063541 1919 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 04:48:41.063719 1919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "dcf3c47f2a15418fad08b8ea9d05775c"
format_stamp: "Formatted at 2026-08-09 04:48:41 on dist-test-slave-1jjb"
server_key: "7bdfc0f989796f0d4ee824793ece9d80"
server_key_iv: "2bd0502f9a0d469c73679c438f56b957"
server_key_version: "encryptionkey@0"
I20260809 04:48:41.063910 1919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:41.079288 1919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:41.083357 1919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:41.083654 1919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:41.084581 1919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:41.085811 1919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:48:41.085908 1919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:41.086054 1919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:41.086141 1919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:41.118901 1919 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:35239
I20260809 04:48:41.118980 2055 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:35239 every 8 connection(s)
I20260809 04:48:41.120127 1919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:48:41.128762 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 1919
I20260809 04:48:41.128950 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance
I20260809 04:48:41.129545 13224 external_mini_cluster.cc:1428] Setting key 51f5ead3a353452764c20e5314e4b7aa
Aug 09 04:48:41 dist-test-slave-1jjb krb5kdc[1800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250921, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.1@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:48:41.139091 2058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:41.122538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:43639 (local address 127.12.234.62:44411)
0809 04:48:41.123007 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:41.123015 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:41.123572 (+ 557us) server_negotiation.cc:407] Connection header received
0809 04:48:41.125298 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:41.125303 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:41.125388 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:41.125497 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:41.126750 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:41.127407 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:41.129403 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:41.129894 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:41.129992 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:41.133619 (+ 3627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:41.133628 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:41.133632 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:41.133668 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:41.136276 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:41.136943 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:41.136948 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:41.136951 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:41.137068 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:41.137471 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:41.137475 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:41.137478 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:41.137738 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:41.137888 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:41.138468 (+ 580us) server_negotiation.cc:299] Negotiation successful
0809 04:48:41.138716 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":238,"threads_started":1}
I20260809 04:48:41.139952 2059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:41.122775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44411 (local address 127.12.234.1:43639)
0809 04:48:41.123415 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:41.123458 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:41.125041 (+ 1583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:41.125711 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:41.125718 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:41.126135 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:48:41.126590 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:41.126605 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:41.127568 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:41.127572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:41.129238 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:41.129257 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:41.130129 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:41.130135 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:41.130330 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:41.130935 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:48:41.130956 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:48:41.133366 (+ 2410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:48:41.136430 (+ 3064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:41.136438 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:41.136450 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:41.136824 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:41.137173 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:41.137178 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:41.137181 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:41.137358 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:41.137862 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:48:41.137868 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:48:41.138157 (+ 289us) client_negotiation.cc:769] Sending connection context
0809 04:48:41.138383 (+ 226us) client_negotiation.cc:240] Negotiation successful
0809 04:48:41.138838 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":259,"threads_started":1}
I20260809 04:48:41.141299 2056 heartbeater.cc:347] Connected to a master server at 127.12.234.62:44411
I20260809 04:48:41.141551 2056 heartbeater.cc:464] Registering TS with master...
I20260809 04:48:41.142149 2056 heartbeater.cc:511] Master 127.12.234.62:44411 requested a full tablet report, sending...
I20260809 04:48:41.143966 1852 ts_manager.cc:194] Registered new tserver with Master: dcf3c47f2a15418fad08b8ea9d05775c (127.12.234.1:35239)
I20260809 04:48:41.145321 1852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.1@KRBTEST.COM'} at 127.12.234.1:43639
WARNING: no policy specified for kudu/127.12.234.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
Entry for principal kudu/127.12.234.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab.
I20260809 04:48:41.191172 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:0
--local_ip_for_outbound_sockets=127.12.234.2
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44411
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:40221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:48:41.503953 2063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:41.504267 2063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:41.504369 2063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:41.510919 2063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:41.511010 2063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:41.511135 2063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:48:41.520638 2063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:40221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.2.keytab
--principal=kudu/127.12.234.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44411
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:41.521637 2063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:41.522869 2063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:41.535820 2068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:41.535899 2071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:41.535859 2069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:41.536958 2063 server_base.cc:1034] running on GCE node
I20260809 04:48:41.537561 2063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:41.538379 2063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:41.539613 2063 hybrid_clock.cc:648] HybridClock initialized: now 1786250921539552 us; error 42 us; skew 500 ppm
I20260809 04:48:41.540024 2063 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:48:41 dist-test-slave-1jjb krb5kdc[1800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250921, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:41.543054 2063 init.cc:377] Logged in from keytab as kudu/127.12.234.2@KRBTEST.COM (short username kudu)
I20260809 04:48:41.544443 2063 webserver.cc:460] Webserver started at http://127.12.234.2:43795/ using document root <none> and password file <none>
I20260809 04:48:41.545042 2063 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:41.545159 2063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:41.545481 2063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:41.548009 2063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-1/data/instance:
uuid: "9acf878500bd47409e9b746a98d14ec5"
format_stamp: "Formatted at 2026-08-09 04:48:41 on dist-test-slave-1jjb"
server_key: "a0fda576fe8230cbb09601dcc8266c1a"
server_key_iv: "38802a556822d9cb57fc2206255e5904"
server_key_version: "encryptionkey@0"
I20260809 04:48:41.548843 2063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance:
uuid: "9acf878500bd47409e9b746a98d14ec5"
format_stamp: "Formatted at 2026-08-09 04:48:41 on dist-test-slave-1jjb"
server_key: "a0fda576fe8230cbb09601dcc8266c1a"
server_key_iv: "38802a556822d9cb57fc2206255e5904"
server_key_version: "encryptionkey@0"
I20260809 04:48:41.553864 2063 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 04:48:41.557050 2080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:41.558225 2063 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:48:41.558375 2063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "9acf878500bd47409e9b746a98d14ec5"
format_stamp: "Formatted at 2026-08-09 04:48:41 on dist-test-slave-1jjb"
server_key: "a0fda576fe8230cbb09601dcc8266c1a"
server_key_iv: "38802a556822d9cb57fc2206255e5904"
server_key_version: "encryptionkey@0"
I20260809 04:48:41.558566 2063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:41.575693 2063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:41.579154 2063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:41.579427 2063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:41.580359 2063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:41.581467 2063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:48:41.581542 2063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:41.581691 2063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:41.581736 2063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:41.611688 2063 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:35663
I20260809 04:48:41.611759 2201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:35663 every 8 connection(s)
I20260809 04:48:41.612859 2063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:48:41.614040 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 2063
I20260809 04:48:41.614193 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance
I20260809 04:48:41.614619 13224 external_mini_cluster.cc:1428] Setting key 8ad78f5cd4a81ae19abc2bf6e20c4630
Aug 09 04:48:41 dist-test-slave-1jjb krb5kdc[1800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250921, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.2@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:48:41.633178 2058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:41.616553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:43319 (local address 127.12.234.62:44411)
0809 04:48:41.616731 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:41.616736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:41.617971 (+ 1235us) server_negotiation.cc:407] Connection header received
0809 04:48:41.619688 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:41.619693 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:41.619775 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:41.619910 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:41.621291 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:41.621989 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:41.623420 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:41.623846 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:41.623977 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:41.627475 (+ 3498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:41.627480 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:41.627483 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:41.627513 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:41.630371 (+ 2858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:41.631027 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:41.631030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:41.631032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:41.631140 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:41.631583 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:41.631586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:41.631587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:41.631865 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:41.631973 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:41.632774 (+ 801us) server_negotiation.cc:299] Negotiation successful
0809 04:48:41.632945 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:48:41.634460 2204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:41.617123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44411 (local address 127.12.234.2:43319)
0809 04:48:41.617828 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:41.617866 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:41.619425 (+ 1559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:41.620187 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:41.620195 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:41.620630 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:48:41.621131 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:41.621146 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:41.622133 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:41.622142 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:41.623275 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:41.623287 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:41.624106 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:41.624115 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:41.624355 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:41.625036 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:48:41.625062 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:48:41.627283 (+ 2221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:48:41.630498 (+ 3215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:41.630503 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:41.630512 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:41.630888 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:41.631241 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:41.631247 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:41.631252 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:41.631466 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:41.631989 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:48:41.631999 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:48:41.632410 (+ 411us) client_negotiation.cc:769] Sending connection context
0809 04:48:41.632685 (+ 275us) client_negotiation.cc:240] Negotiation successful
0809 04:48:41.633215 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":302,"threads_started":1}
I20260809 04:48:41.635731 2202 heartbeater.cc:347] Connected to a master server at 127.12.234.62:44411
I20260809 04:48:41.635987 2202 heartbeater.cc:464] Registering TS with master...
I20260809 04:48:41.636727 2202 heartbeater.cc:511] Master 127.12.234.62:44411 requested a full tablet report, sending...
I20260809 04:48:41.638115 1852 ts_manager.cc:194] Registered new tserver with Master: 9acf878500bd47409e9b746a98d14ec5 (127.12.234.2:35663)
I20260809 04:48:41.639385 1852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.2@KRBTEST.COM'} at 127.12.234.2:43319
WARNING: no policy specified for kudu/127.12.234.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
Entry for principal kudu/127.12.234.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab.
I20260809 04:48:41.681384 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:0
--local_ip_for_outbound_sockets=127.12.234.3
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44411
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:40221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:48:41.984738 2207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:41.984983 2207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:41.985071 2207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:41.991536 2207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:41.991621 2207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:41.991770 2207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:48:41.999894 2207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:40221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.3.keytab
--principal=kudu/127.12.234.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:44411
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:42.001021 2207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:42.002131 2207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:42.013866 2213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:42.013924 2212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:42.013873 2215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:42.014788 2207 server_base.cc:1034] running on GCE node
I20260809 04:48:42.015224 2207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:42.015942 2207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:42.017135 2207 hybrid_clock.cc:648] HybridClock initialized: now 1786250922017084 us; error 45 us; skew 500 ppm
I20260809 04:48:42.017374 2207 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:48:42 dist-test-slave-1jjb krb5kdc[1800](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250922, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:42.020311 2207 init.cc:377] Logged in from keytab as kudu/127.12.234.3@KRBTEST.COM (short username kudu)
I20260809 04:48:42.021668 2207 webserver.cc:460] Webserver started at http://127.12.234.3:36919/ using document root <none> and password file <none>
I20260809 04:48:42.022300 2207 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:42.022445 2207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:42.022733 2207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:42.025519 2207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-2/data/instance:
uuid: "ee4a46cdc1994812b5831a226d471135"
format_stamp: "Formatted at 2026-08-09 04:48:42 on dist-test-slave-1jjb"
server_key: "e76b8898843e11df9e0c29d5d5c2d339"
server_key_iv: "cec6e4dc96cf9eb9a4e1a9a2ebb61121"
server_key_version: "encryptionkey@0"
I20260809 04:48:42.026207 2207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance:
uuid: "ee4a46cdc1994812b5831a226d471135"
format_stamp: "Formatted at 2026-08-09 04:48:42 on dist-test-slave-1jjb"
server_key: "e76b8898843e11df9e0c29d5d5c2d339"
server_key_iv: "cec6e4dc96cf9eb9a4e1a9a2ebb61121"
server_key_version: "encryptionkey@0"
I20260809 04:48:42.030874 2207 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:48:42.033981 2223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:42.035120 2207 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:48:42.035301 2207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "ee4a46cdc1994812b5831a226d471135"
format_stamp: "Formatted at 2026-08-09 04:48:42 on dist-test-slave-1jjb"
server_key: "e76b8898843e11df9e0c29d5d5c2d339"
server_key_iv: "cec6e4dc96cf9eb9a4e1a9a2ebb61121"
server_key_version: "encryptionkey@0"
I20260809 04:48:42.035517 2207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:42.086860 2207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:42.091293 2207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:42.091594 2207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:42.092634 2207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:42.093909 2207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:48:42.094008 2207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:42.094144 2207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:42.094211 2207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:42.127563 2207 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:46015
I20260809 04:48:42.127614 2342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:46015 every 8 connection(s)
I20260809 04:48:42.128827 2207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:48:42.135231 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 2207
I20260809 04:48:42.135443 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance
I20260809 04:48:42.135877 13224 external_mini_cluster.cc:1428] Setting key cd41a2b2ae143bf5b42603ffffe8f913
Aug 09 04:48:42 dist-test-slave-1jjb krb5kdc[1800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250922, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.3@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:48:42.147888 2056 heartbeater.cc:503] Master 127.12.234.62:44411 was elected leader, sending a full tablet report...
I20260809 04:48:42.149422 2058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:42.131752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:41191 (local address 127.12.234.62:44411)
0809 04:48:42.131934 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:42.131939 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:42.133161 (+ 1222us) server_negotiation.cc:407] Connection header received
0809 04:48:42.134918 (+ 1757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:42.134923 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:42.135021 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:42.135125 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:42.136672 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.137553 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.139178 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.139770 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.139924 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:42.143193 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:42.143198 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:42.143202 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:42.143241 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:42.145978 (+ 2737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:42.146717 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:42.146722 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:42.146724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:42.146856 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:42.147306 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:42.147309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:42.147311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:42.147572 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:42.147727 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:42.148829 (+ 1102us) server_negotiation.cc:299] Negotiation successful
0809 04:48:42.149024 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:48:42.150477 2345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:42.132268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:44411 (local address 127.12.234.3:41191)
0809 04:48:42.132996 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:42.133034 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:42.134645 (+ 1611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:42.135345 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:42.135372 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:42.135894 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:48:42.136512 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:42.136529 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.137692 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.137697 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:42.139020 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:42.139033 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.140009 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.140019 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:42.140300 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:42.140959 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:48:42.140980 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:48:42.143004 (+ 2024us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:48:42.146141 (+ 3137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:42.146150 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:42.146166 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:42.146585 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:42.146979 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:48:42.146984 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:48:42.146988 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:48:42.147183 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:48:42.147705 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:48:42.147712 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:48:42.148130 (+ 418us) client_negotiation.cc:769] Sending connection context
0809 04:48:42.148433 (+ 303us) client_negotiation.cc:240] Negotiation successful
0809 04:48:42.149036 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":318,"threads_started":1}
I20260809 04:48:42.151899 2343 heartbeater.cc:347] Connected to a master server at 127.12.234.62:44411
I20260809 04:48:42.152259 2343 heartbeater.cc:464] Registering TS with master...
I20260809 04:48:42.153031 2343 heartbeater.cc:511] Master 127.12.234.62:44411 requested a full tablet report, sending...
I20260809 04:48:42.154457 1851 ts_manager.cc:194] Registered new tserver with Master: ee4a46cdc1994812b5831a226d471135 (127.12.234.3:46015)
I20260809 04:48:42.155345 1851 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.234.3@KRBTEST.COM'} at 127.12.234.3:41191
I20260809 04:48:42.164212 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:48:42.180871 2058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:42.168368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33486 (local address 127.12.234.62:44411)
0809 04:48:42.168656 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:42.168666 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:42.168982 (+ 316us) server_negotiation.cc:407] Connection header received
0809 04:48:42.169122 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:42.169125 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:42.169191 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:42.169271 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:42.170686 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.171419 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.172765 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.173172 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.173264 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:42.175773 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:42.175779 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:42.175782 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:42.175811 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:42.178342 (+ 2531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:42.179025 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:42.179029 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:42.179031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:42.179164 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:42.179506 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:42.179509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:42.179511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:42.179839 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:42.180046 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:42.180391 (+ 345us) server_negotiation.cc:299] Negotiation successful
0809 04:48:42.180570 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 04:48:42.185192 1851 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33486:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:48:42.192525 1851 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:48:42.215986 2357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:42.205663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41502 (local address 127.12.234.2:35663)
0809 04:48:42.206342 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:42.206351 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:42.206656 (+ 305us) server_negotiation.cc:407] Connection header received
0809 04:48:42.207030 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:42.207039 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:42.207309 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:42.208275 (+ 966us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:42.208534 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.210127 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.213204 (+ 3077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.214784 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.214948 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:42.215169 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:42.215465 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 04:48:42.215656 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":365,"threads_started":1}
I20260809 04:48:42.216728 2356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:42.205543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:35663 (local address 127.0.0.1:41502)
0809 04:48:42.206492 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:42.206533 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:42.206756 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:42.207571 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:42.207580 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:42.207614 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:42.208003 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:42.208021 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.210279 (+ 2258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.210287 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:42.212997 (+ 2710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:42.213014 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.214979 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.214985 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:42.215187 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:42.215245 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 04:48:42.215394 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 04:48:42.215642 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"mutex_wait_us":36,"thread_start_us":582,"threads_started":1}
I20260809 04:48:42.217527 2360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:42.207402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:46015 (local address 127.0.0.1:49648)
0809 04:48:42.208349 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:42.208373 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:42.208496 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:42.209825 (+ 1329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:42.209830 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:42.209858 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:42.210696 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:42.210714 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.212810 (+ 2096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.212816 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:42.214524 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:42.214537 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.216544 (+ 2007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.216549 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:42.216792 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:42.216829 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 04:48:42.216915 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 04:48:42.217070 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"mutex_wait_us":120,"spinlock_wait_cycles":176384,"thread_start_us":1209,"threads_started":1}
I20260809 04:48:42.217916 2358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:42.206792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:35239 (local address 127.0.0.1:39462)
0809 04:48:42.207331 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:42.207350 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:42.207468 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:42.209909 (+ 2441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:42.209918 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:42.209977 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:42.210661 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:42.210677 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.212195 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.212200 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:42.214571 (+ 2371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:42.214587 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.216539 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.216546 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:42.217153 (+ 607us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:42.217200 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 04:48:42.217286 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 04:48:42.217413 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"mutex_wait_us":450,"thread_start_us":288,"threads_started":1}
I20260809 04:48:42.218163 2359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:42.207226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39462 (local address 127.12.234.1:35239)
0809 04:48:42.209342 (+ 2116us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:42.209350 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:42.209380 (+ 30us) server_negotiation.cc:407] Connection header received
0809 04:48:42.209473 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:42.209480 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:42.209701 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:42.210350 (+ 649us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:42.210856 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.212018 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.214755 (+ 2737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.216231 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.216819 (+ 588us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:42.217077 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:42.217639 (+ 562us) server_negotiation.cc:299] Negotiation successful
0809 04:48:42.217839 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":468,"threads_started":1}
I20260809 04:48:42.218978 2361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:42.208045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49648 (local address 127.12.234.3:46015)
0809 04:48:42.209292 (+ 1247us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:42.209301 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:42.209336 (+ 35us) server_negotiation.cc:407] Connection header received
0809 04:48:42.209438 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:42.209444 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:42.209636 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:42.209834 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:42.210939 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.212632 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.214676 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.216233 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.217951 (+ 1718us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:42.218377 (+ 426us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:42.218454 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 04:48:42.218638 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1070,"thread_start_us":1151,"threads_started":1}
I20260809 04:48:42.221321 2130 tablet_service.cc:1467] Processing CreateTablet for tablet 9d837da8711045caaba8f7682001e36d (DEFAULT_TABLE table=test-table [id=b23030146ecd4231a4a50edd40f29720]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:48:42.221566 1988 tablet_service.cc:1467] Processing CreateTablet for tablet 9d837da8711045caaba8f7682001e36d (DEFAULT_TABLE table=test-table [id=b23030146ecd4231a4a50edd40f29720]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:48:42.222874 1988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d837da8711045caaba8f7682001e36d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:42.222947 2130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d837da8711045caaba8f7682001e36d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:42.222496 2273 tablet_service.cc:1467] Processing CreateTablet for tablet 9d837da8711045caaba8f7682001e36d (DEFAULT_TABLE table=test-table [id=b23030146ecd4231a4a50edd40f29720]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:48:42.223860 2273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d837da8711045caaba8f7682001e36d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:42.235285 2362 tablet_bootstrap.cc:492] T 9d837da8711045caaba8f7682001e36d P dcf3c47f2a15418fad08b8ea9d05775c: Bootstrap starting.
I20260809 04:48:42.239563 2363 tablet_bootstrap.cc:492] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5: Bootstrap starting.
I20260809 04:48:42.240383 2362 tablet_bootstrap.cc:654] T 9d837da8711045caaba8f7682001e36d P dcf3c47f2a15418fad08b8ea9d05775c: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:42.241930 2362 log.cc:824] T 9d837da8711045caaba8f7682001e36d P dcf3c47f2a15418fad08b8ea9d05775c: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:42.243592 2364 tablet_bootstrap.cc:492] T 9d837da8711045caaba8f7682001e36d P ee4a46cdc1994812b5831a226d471135: Bootstrap starting.
I20260809 04:48:42.244203 2363 tablet_bootstrap.cc:654] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:42.245307 2362 tablet_bootstrap.cc:492] T 9d837da8711045caaba8f7682001e36d P dcf3c47f2a15418fad08b8ea9d05775c: No bootstrap required, opened a new log
I20260809 04:48:42.245597 2362 ts_tablet_manager.cc:1397] T 9d837da8711045caaba8f7682001e36d P dcf3c47f2a15418fad08b8ea9d05775c: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260809 04:48:42.245781 2363 log.cc:824] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:42.247411 2364 tablet_bootstrap.cc:654] T 9d837da8711045caaba8f7682001e36d P ee4a46cdc1994812b5831a226d471135: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:42.248831 2364 log.cc:824] T 9d837da8711045caaba8f7682001e36d P ee4a46cdc1994812b5831a226d471135: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:42.249279 2363 tablet_bootstrap.cc:492] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5: No bootstrap required, opened a new log
I20260809 04:48:42.249709 2363 ts_tablet_manager.cc:1397] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260809 04:48:42.254200 2362 raft_consensus.cc:348] T 9d837da8711045caaba8f7682001e36d P dcf3c47f2a15418fad08b8ea9d05775c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9acf878500bd47409e9b746a98d14ec5" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 35663 } } peers { permanent_uuid: "dcf3c47f2a15418fad08b8ea9d05775c" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 35239 } } peers { permanent_uuid: "ee4a46cdc1994812b5831a226d471135" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 46015 } }
I20260809 04:48:42.254601 2362 raft_consensus.cc:374] T 9d837da8711045caaba8f7682001e36d P dcf3c47f2a15418fad08b8ea9d05775c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:42.254760 2362 raft_consensus.cc:729] T 9d837da8711045caaba8f7682001e36d P dcf3c47f2a15418fad08b8ea9d05775c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcf3c47f2a15418fad08b8ea9d05775c, State: Initialized, Role: FOLLOWER
I20260809 04:48:42.255342 2362 consensus_queue.cc:260] T 9d837da8711045caaba8f7682001e36d P dcf3c47f2a15418fad08b8ea9d05775c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9acf878500bd47409e9b746a98d14ec5" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 35663 } } peers { permanent_uuid: "dcf3c47f2a15418fad08b8ea9d05775c" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 35239 } } peers { permanent_uuid: "ee4a46cdc1994812b5831a226d471135" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 46015 } }
I20260809 04:48:42.257257 2362 ts_tablet_manager.cc:1428] T 9d837da8711045caaba8f7682001e36d P dcf3c47f2a15418fad08b8ea9d05775c: Time spent starting tablet: real 0.012s user 0.004s sys 0.008s
I20260809 04:48:42.260866 2364 tablet_bootstrap.cc:492] T 9d837da8711045caaba8f7682001e36d P ee4a46cdc1994812b5831a226d471135: No bootstrap required, opened a new log
I20260809 04:48:42.261237 2364 ts_tablet_manager.cc:1397] T 9d837da8711045caaba8f7682001e36d P ee4a46cdc1994812b5831a226d471135: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.011s
I20260809 04:48:42.262655 2363 raft_consensus.cc:348] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9acf878500bd47409e9b746a98d14ec5" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 35663 } } peers { permanent_uuid: "dcf3c47f2a15418fad08b8ea9d05775c" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 35239 } } peers { permanent_uuid: "ee4a46cdc1994812b5831a226d471135" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 46015 } }
I20260809 04:48:42.263089 2363 raft_consensus.cc:374] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:42.263212 2363 raft_consensus.cc:729] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9acf878500bd47409e9b746a98d14ec5, State: Initialized, Role: FOLLOWER
I20260809 04:48:42.263873 2363 consensus_queue.cc:260] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9acf878500bd47409e9b746a98d14ec5" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 35663 } } peers { permanent_uuid: "dcf3c47f2a15418fad08b8ea9d05775c" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 35239 } } peers { permanent_uuid: "ee4a46cdc1994812b5831a226d471135" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 46015 } }
I20260809 04:48:42.265173 2202 heartbeater.cc:503] Master 127.12.234.62:44411 was elected leader, sending a full tablet report...
I20260809 04:48:42.265859 2363 ts_tablet_manager.cc:1428] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5: Time spent starting tablet: real 0.016s user 0.014s sys 0.001s
I20260809 04:48:42.270916 2364 raft_consensus.cc:348] T 9d837da8711045caaba8f7682001e36d P ee4a46cdc1994812b5831a226d471135 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9acf878500bd47409e9b746a98d14ec5" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 35663 } } peers { permanent_uuid: "dcf3c47f2a15418fad08b8ea9d05775c" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 35239 } } peers { permanent_uuid: "ee4a46cdc1994812b5831a226d471135" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 46015 } }
I20260809 04:48:42.271345 2364 raft_consensus.cc:374] T 9d837da8711045caaba8f7682001e36d P ee4a46cdc1994812b5831a226d471135 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:42.271458 2364 raft_consensus.cc:729] T 9d837da8711045caaba8f7682001e36d P ee4a46cdc1994812b5831a226d471135 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee4a46cdc1994812b5831a226d471135, State: Initialized, Role: FOLLOWER
I20260809 04:48:42.272113 2364 consensus_queue.cc:260] T 9d837da8711045caaba8f7682001e36d P ee4a46cdc1994812b5831a226d471135 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9acf878500bd47409e9b746a98d14ec5" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 35663 } } peers { permanent_uuid: "dcf3c47f2a15418fad08b8ea9d05775c" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 35239 } } peers { permanent_uuid: "ee4a46cdc1994812b5831a226d471135" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 46015 } }
I20260809 04:48:42.273358 2343 heartbeater.cc:503] Master 127.12.234.62:44411 was elected leader, sending a full tablet report...
I20260809 04:48:42.274161 2364 ts_tablet_manager.cc:1428] T 9d837da8711045caaba8f7682001e36d P ee4a46cdc1994812b5831a226d471135: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
W20260809 04:48:42.364737 2203 tablet.cc:2369] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:48:42.372488 2057 tablet.cc:2369] T 9d837da8711045caaba8f7682001e36d P dcf3c47f2a15418fad08b8ea9d05775c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:48:42.380443 2344 tablet.cc:2369] T 9d837da8711045caaba8f7682001e36d P ee4a46cdc1994812b5831a226d471135: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:48:42.517642 2369 raft_consensus.cc:482] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:48:42.517998 2369 raft_consensus.cc:504] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9acf878500bd47409e9b746a98d14ec5" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 35663 } } peers { permanent_uuid: "dcf3c47f2a15418fad08b8ea9d05775c" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 35239 } } peers { permanent_uuid: "ee4a46cdc1994812b5831a226d471135" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 46015 } }
I20260809 04:48:42.519682 2369 leader_election.cc:288] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dcf3c47f2a15418fad08b8ea9d05775c (127.12.234.1:35239), ee4a46cdc1994812b5831a226d471135 (127.12.234.3:46015)
I20260809 04:48:42.526633 2361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:42.521021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:52409 (local address 127.12.234.3:46015)
0809 04:48:42.521214 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:42.521221 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:42.521518 (+ 297us) server_negotiation.cc:407] Connection header received
0809 04:48:42.521665 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:42.521671 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:42.521754 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:42.521877 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:42.522496 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.523223 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.524953 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.525705 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.525873 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:42.525966 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:42.526246 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 04:48:42.526380 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 04:48:42.526635 2359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:42.520121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:59605 (local address 127.12.234.1:35239)
0809 04:48:42.520374 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:42.520383 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:42.520909 (+ 526us) server_negotiation.cc:407] Connection header received
0809 04:48:42.521154 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:42.521160 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:42.521243 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:42.521410 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:42.522417 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.523125 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.525019 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.525742 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.525917 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:42.525983 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:42.526322 (+ 339us) server_negotiation.cc:299] Negotiation successful
0809 04:48:42.526443 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 04:48:42.526700 2372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:42.520788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:46015 (local address 127.12.234.2:52409)
0809 04:48:42.521402 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:42.521416 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:42.521530 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:42.521895 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:42.521901 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:42.521919 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:42.522292 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:42.522301 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.523354 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.523357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:42.524796 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:42.524807 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.525931 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.525937 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:42.526097 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:42.526118 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 04:48:42.526185 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 04:48:42.526310 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"mutex_wait_us":78,"thread_start_us":300,"threads_started":1}
I20260809 04:48:42.526810 2371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:42.520103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:35239 (local address 127.12.234.2:59605)
0809 04:48:42.520769 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:42.520791 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:42.520952 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:42.521668 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:42.521674 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:42.521703 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:42.522222 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:42.522236 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.523262 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.523270 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:42.524834 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:42.524847 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.525991 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.526003 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:42.526138 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:42.526160 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 04:48:42.526287 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 04:48:42.526401 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"mutex_wait_us":18,"thread_start_us":380,"threads_started":1}
I20260809 04:48:42.527657 2010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d837da8711045caaba8f7682001e36d" candidate_uuid: "9acf878500bd47409e9b746a98d14ec5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcf3c47f2a15418fad08b8ea9d05775c" is_pre_election: true
I20260809 04:48:42.527688 2297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d837da8711045caaba8f7682001e36d" candidate_uuid: "9acf878500bd47409e9b746a98d14ec5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee4a46cdc1994812b5831a226d471135" is_pre_election: true
I20260809 04:48:42.528149 2010 raft_consensus.cc:2454] T 9d837da8711045caaba8f7682001e36d P dcf3c47f2a15418fad08b8ea9d05775c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9acf878500bd47409e9b746a98d14ec5 in term 0.
I20260809 04:48:42.528158 2297 raft_consensus.cc:2454] T 9d837da8711045caaba8f7682001e36d P ee4a46cdc1994812b5831a226d471135 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9acf878500bd47409e9b746a98d14ec5 in term 0.
I20260809 04:48:42.528762 2084 leader_election.cc:302] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9acf878500bd47409e9b746a98d14ec5, ee4a46cdc1994812b5831a226d471135; no voters:
I20260809 04:48:42.529134 2369 raft_consensus.cc:2781] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:48:42.529315 2369 raft_consensus.cc:482] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:48:42.529435 2369 raft_consensus.cc:3037] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:42.531776 2369 raft_consensus.cc:504] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9acf878500bd47409e9b746a98d14ec5" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 35663 } } peers { permanent_uuid: "dcf3c47f2a15418fad08b8ea9d05775c" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 35239 } } peers { permanent_uuid: "ee4a46cdc1994812b5831a226d471135" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 46015 } }
I20260809 04:48:42.532384 2369 leader_election.cc:288] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5 [CANDIDATE]: Term 1 election: Requested vote from peers dcf3c47f2a15418fad08b8ea9d05775c (127.12.234.1:35239), ee4a46cdc1994812b5831a226d471135 (127.12.234.3:46015)
I20260809 04:48:42.532737 2010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d837da8711045caaba8f7682001e36d" candidate_uuid: "9acf878500bd47409e9b746a98d14ec5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcf3c47f2a15418fad08b8ea9d05775c"
I20260809 04:48:42.532876 2297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d837da8711045caaba8f7682001e36d" candidate_uuid: "9acf878500bd47409e9b746a98d14ec5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee4a46cdc1994812b5831a226d471135"
I20260809 04:48:42.532966 2010 raft_consensus.cc:3037] T 9d837da8711045caaba8f7682001e36d P dcf3c47f2a15418fad08b8ea9d05775c [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:42.533075 2297 raft_consensus.cc:3037] T 9d837da8711045caaba8f7682001e36d P ee4a46cdc1994812b5831a226d471135 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:42.535938 2297 raft_consensus.cc:2454] T 9d837da8711045caaba8f7682001e36d P ee4a46cdc1994812b5831a226d471135 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9acf878500bd47409e9b746a98d14ec5 in term 1.
I20260809 04:48:42.535938 2010 raft_consensus.cc:2454] T 9d837da8711045caaba8f7682001e36d P dcf3c47f2a15418fad08b8ea9d05775c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9acf878500bd47409e9b746a98d14ec5 in term 1.
I20260809 04:48:42.536326 2084 leader_election.cc:302] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9acf878500bd47409e9b746a98d14ec5, dcf3c47f2a15418fad08b8ea9d05775c; no voters:
I20260809 04:48:42.536585 2369 raft_consensus.cc:2781] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:48:42.537021 2369 raft_consensus.cc:686] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5 [term 1 LEADER]: Becoming Leader. State: Replica: 9acf878500bd47409e9b746a98d14ec5, State: Running, Role: LEADER
I20260809 04:48:42.537480 2369 consensus_queue.cc:237] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9acf878500bd47409e9b746a98d14ec5" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 35663 } } peers { permanent_uuid: "dcf3c47f2a15418fad08b8ea9d05775c" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 35239 } } peers { permanent_uuid: "ee4a46cdc1994812b5831a226d471135" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 46015 } }
I20260809 04:48:42.542202 1851 catalog_manager.cc:5495] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9acf878500bd47409e9b746a98d14ec5 (127.12.234.2). New cstate: current_term: 1 leader_uuid: "9acf878500bd47409e9b746a98d14ec5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9acf878500bd47409e9b746a98d14ec5" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 35663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dcf3c47f2a15418fad08b8ea9d05775c" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 35239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee4a46cdc1994812b5831a226d471135" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 46015 } health_report { overall_health: UNKNOWN } } }
I20260809 04:48:42.586246 2357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:42.581321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41512 (local address 127.12.234.2:35663)
0809 04:48:42.581511 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:42.581515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:42.581532 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:48:42.581756 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:42.581760 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:42.581841 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:42.581975 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:48:42.582299 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.583062 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.584486 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.584863 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.584966 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:42.585352 (+ 386us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:48:42.585811 (+ 459us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:48:42.585934 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:42.585979 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 04:48:42.586037 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 04:48:42.594383 2010 raft_consensus.cc:1261] T 9d837da8711045caaba8f7682001e36d P dcf3c47f2a15418fad08b8ea9d05775c [term 1 FOLLOWER]: Refusing update from remote peer 9acf878500bd47409e9b746a98d14ec5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:48:42.594381 2297 raft_consensus.cc:1261] T 9d837da8711045caaba8f7682001e36d P ee4a46cdc1994812b5831a226d471135 [term 1 FOLLOWER]: Refusing update from remote peer 9acf878500bd47409e9b746a98d14ec5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:48:42.595338 2373 consensus_queue.cc:1035] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcf3c47f2a15418fad08b8ea9d05775c" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 35239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:48:42.595688 2369 consensus_queue.cc:1035] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee4a46cdc1994812b5831a226d471135" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 46015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:48:42.605563 2376 mvcc.cc:205] Tried to move back new op lower bound from 7316483778935676928 to 7316483778717429760. Current Snapshot: MvccSnapshot[applied={T|T < 7316483778935676928}]
I20260809 04:48:42.620419 1852 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33486:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:48:42.620668 1852 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33486:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:48:42.624444 1852 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c2c8e1f1c984cd48e5598baa6d82bfc: Sending DeleteTablet for 3 replicas of tablet 9d837da8711045caaba8f7682001e36d
I20260809 04:48:42.625630 2273 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d837da8711045caaba8f7682001e36d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:48:42 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:49648
I20260809 04:48:42.625643 1988 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d837da8711045caaba8f7682001e36d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:48:42 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:39462
I20260809 04:48:42.626801 2130 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d837da8711045caaba8f7682001e36d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:48:42 UTC) from {username='kudu', principal='kudu/127.12.234.62@KRBTEST.COM'} at 127.0.0.1:41502
I20260809 04:48:42.628226 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 1919
I20260809 04:48:42.636534 2388 tablet_replica.cc:330] T 9d837da8711045caaba8f7682001e36d P dcf3c47f2a15418fad08b8ea9d05775c: stopping tablet replica
I20260809 04:48:42.636994 2388 raft_consensus.cc:2229] T 9d837da8711045caaba8f7682001e36d P dcf3c47f2a15418fad08b8ea9d05775c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:48:42.637488 2388 raft_consensus.cc:2258] T 9d837da8711045caaba8f7682001e36d P dcf3c47f2a15418fad08b8ea9d05775c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:48:42.639835 2388 ts_tablet_manager.cc:1905] T 9d837da8711045caaba8f7682001e36d P dcf3c47f2a15418fad08b8ea9d05775c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:48:42.646150 2388 ts_tablet_manager.cc:1918] T 9d837da8711045caaba8f7682001e36d P dcf3c47f2a15418fad08b8ea9d05775c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:48:42.646339 2388 log.cc:1196] T 9d837da8711045caaba8f7682001e36d P dcf3c47f2a15418fad08b8ea9d05775c: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-0/wal/wals/9d837da8711045caaba8f7682001e36d
I20260809 04:48:42.646904 2388 ts_tablet_manager.cc:1939] T 9d837da8711045caaba8f7682001e36d P dcf3c47f2a15418fad08b8ea9d05775c: Deleting consensus metadata
I20260809 04:48:42.648602 1837 catalog_manager.cc:4841] TS dcf3c47f2a15418fad08b8ea9d05775c (127.12.234.1:35239): tablet 9d837da8711045caaba8f7682001e36d (table test-table [id=b23030146ecd4231a4a50edd40f29720]) successfully deleted
I20260809 04:48:42.657131 2389 tablet_replica.cc:330] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5: stopping tablet replica
I20260809 04:48:42.657564 2389 raft_consensus.cc:2229] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:48:42.658190 2389 raft_consensus.cc:2258] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:48:42.660552 2387 tablet_replica.cc:330] T 9d837da8711045caaba8f7682001e36d P ee4a46cdc1994812b5831a226d471135: stopping tablet replica
I20260809 04:48:42.660905 2387 raft_consensus.cc:2229] T 9d837da8711045caaba8f7682001e36d P ee4a46cdc1994812b5831a226d471135 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:48:42.661343 2387 raft_consensus.cc:2258] T 9d837da8711045caaba8f7682001e36d P ee4a46cdc1994812b5831a226d471135 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:48:42.661823 2389 ts_tablet_manager.cc:1905] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:48:42.665510 2387 ts_tablet_manager.cc:1905] T 9d837da8711045caaba8f7682001e36d P ee4a46cdc1994812b5831a226d471135: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 04:48:42 dist-test-slave-1jjb krb5kdc[1800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.1@KRBTEST.COM
I20260809 04:48:42.669273 2389 ts_tablet_manager.cc:1918] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:48:42.669456 2389 log.cc:1196] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-1/wal/wals/9d837da8711045caaba8f7682001e36d
I20260809 04:48:42.670096 2389 ts_tablet_manager.cc:1939] T 9d837da8711045caaba8f7682001e36d P 9acf878500bd47409e9b746a98d14ec5: Deleting consensus metadata
I20260809 04:48:42.671720 1837 catalog_manager.cc:4841] TS 9acf878500bd47409e9b746a98d14ec5 (127.12.234.2:35663): tablet 9d837da8711045caaba8f7682001e36d (table test-table [id=b23030146ecd4231a4a50edd40f29720]) successfully deleted
I20260809 04:48:42.672911 2387 ts_tablet_manager.cc:1918] T 9d837da8711045caaba8f7682001e36d P ee4a46cdc1994812b5831a226d471135: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:48:42.673065 2387 log.cc:1196] T 9d837da8711045caaba8f7682001e36d P ee4a46cdc1994812b5831a226d471135: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786250603326224-13224-0/minicluster-data/ts-2/wal/wals/9d837da8711045caaba8f7682001e36d
I20260809 04:48:42.673583 2387 ts_tablet_manager.cc:1939] T 9d837da8711045caaba8f7682001e36d P ee4a46cdc1994812b5831a226d471135: Deleting consensus metadata
I20260809 04:48:42.674608 2359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:42.629924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39470 (local address 127.12.234.1:35239)
0809 04:48:42.630151 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:42.630161 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:42.656569 (+ 26408us) server_negotiation.cc:407] Connection header received
0809 04:48:42.656747 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:42.656754 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:42.656843 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:42.656952 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:42.658992 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.660404 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.661863 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.662379 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.662513 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:42.667185 (+ 4672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:42.667199 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:42.667214 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:42.667260 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:42.670985 (+ 3725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:42.671840 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:42.671849 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:42.671851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:42.672110 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:42.672657 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:42.672665 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:42.672671 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:42.673188 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:42.673493 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:42.673823 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 04:48:42.674057 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 04:48:42.675297 1837 catalog_manager.cc:4841] TS ee4a46cdc1994812b5831a226d471135 (127.12.234.3:46015): tablet 9d837da8711045caaba8f7682001e36d (table test-table [id=b23030146ecd4231a4a50edd40f29720]) successfully deleted
I20260809 04:48:42.675712 2051 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:42.760759 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 1919
I20260809 04:48:42.781365 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 2063
Aug 09 04:48:42 dist-test-slave-1jjb krb5kdc[1800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.2@KRBTEST.COM
I20260809 04:48:42.797940 2357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:42.782124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41518 (local address 127.12.234.2:35663)
0809 04:48:42.782292 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:42.782296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:42.782317 (+ 21us) server_negotiation.cc:407] Connection header received
0809 04:48:42.782514 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:42.782517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:42.782595 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:42.782670 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:42.784557 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.785303 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.786875 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.787236 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.787318 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:42.791953 (+ 4635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:42.791964 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:42.791979 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:42.792013 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:42.795225 (+ 3212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:42.795877 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:42.795884 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:42.795886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:42.796026 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:42.796426 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:42.796430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:42.796433 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:42.796800 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:42.797133 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:42.797342 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 04:48:42.797554 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 04:48:42.798677 2197 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:42.901798 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 2063
I20260809 04:48:42.921842 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 2207
Aug 09 04:48:42 dist-test-slave-1jjb krb5kdc[1800](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.3@KRBTEST.COM
I20260809 04:48:42.935472 2361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:42.922568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49660 (local address 127.12.234.3:46015)
0809 04:48:42.922719 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:42.922723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:42.922739 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:48:42.922905 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:42.922909 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:42.922972 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:42.923050 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:42.924290 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.924990 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.926117 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:42.926434 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:42.926512 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:42.930762 (+ 4250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:42.930776 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:42.930792 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:42.930824 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:42.933264 (+ 2440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:42.933799 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:42.933806 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:42.933808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:42.933965 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:42.934279 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:42.934282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:42.934284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:42.934612 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:42.934844 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:42.935076 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 04:48:42.935248 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 04:48:42.936144 2338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:43.016113 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 2207
I20260809 04:48:43.035797 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 1814
I20260809 04:48:43.037048 1886 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:43.139824 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 1814
2026-08-09T04:48:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4492 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T04:48:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:48:43Z Disabled control of system clock
I20260809 04:48:43.228163 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:39683
--webserver_interface=127.12.234.62
--webserver_port=0
--builtin_ntp_servers=127.12.234.20:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:39683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 04:48:43.541357 2398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:43.541590 2398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:43.541659 2398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:43.547830 2398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:48:43.547909 2398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:43.547941 2398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:43.547987 2398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:48:43.548053 2398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:48:43.556151 2398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:39683
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:39683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:43.557096 2398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:43.558502 2398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:43.569407 2407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:43.569408 2404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:43.569923 2403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:43.570310 2398 server_base.cc:1034] running on GCE node
I20260809 04:48:43.570986 2398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:43.572175 2398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:43.573401 2398 hybrid_clock.cc:648] HybridClock initialized: now 1786250923573355 us; error 35 us; skew 500 ppm
I20260809 04:48:43.573704 2398 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:48:43.575721 2398 webserver.cc:460] Webserver started at http://127.12.234.62:39387/ using document root <none> and password file <none>
I20260809 04:48:43.576380 2398 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:43.576496 2398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:43.576815 2398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:43.579445 2398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "5e0a95bb9a894501a886932306e4a79b"
format_stamp: "Formatted at 2026-08-09 04:48:43 on dist-test-slave-1jjb"
server_key: "053487c71d9c56082fa81bf1200f7745"
server_key_iv: "f8f4560a5ed947663bab2f567264e6f7"
server_key_version: "encryptionkey@0"
I20260809 04:48:43.580279 2398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "5e0a95bb9a894501a886932306e4a79b"
format_stamp: "Formatted at 2026-08-09 04:48:43 on dist-test-slave-1jjb"
server_key: "053487c71d9c56082fa81bf1200f7745"
server_key_iv: "f8f4560a5ed947663bab2f567264e6f7"
server_key_version: "encryptionkey@0"
I20260809 04:48:43.584890 2398 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260809 04:48:43.587852 2413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:43.589111 2398 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:48:43.589318 2398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "5e0a95bb9a894501a886932306e4a79b"
format_stamp: "Formatted at 2026-08-09 04:48:43 on dist-test-slave-1jjb"
server_key: "053487c71d9c56082fa81bf1200f7745"
server_key_iv: "f8f4560a5ed947663bab2f567264e6f7"
server_key_version: "encryptionkey@0"
I20260809 04:48:43.589532 2398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:43.601550 2398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:43.602245 2398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:43.602515 2398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:43.619834 2398 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:39683
I20260809 04:48:43.619855 2466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:39683 every 8 connection(s)
I20260809 04:48:43.621045 2398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:48:43.624320 2467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:43.630067 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 2398
I20260809 04:48:43.630231 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:48:43.630666 13224 external_mini_cluster.cc:1428] Setting key 2f1eaded37b67c22058231db0a255d6f
I20260809 04:48:43.634317 2467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e0a95bb9a894501a886932306e4a79b: Bootstrap starting.
I20260809 04:48:43.637840 2467 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e0a95bb9a894501a886932306e4a79b: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:43.638974 2467 log.cc:824] T 00000000000000000000000000000000 P 5e0a95bb9a894501a886932306e4a79b: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:43.641851 2467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e0a95bb9a894501a886932306e4a79b: No bootstrap required, opened a new log
I20260809 04:48:43.644779 2470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:43.632426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51618 (local address 127.12.234.62:39683)
0809 04:48:43.633380 (+ 954us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:43.633393 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:43.633439 (+ 46us) server_negotiation.cc:407] Connection header received
0809 04:48:43.635538 (+ 2099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:43.635562 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:43.636109 (+ 547us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:43.636497 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:43.636945 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:43.638691 (+ 1746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:43.640459 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:43.641326 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:43.641491 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:43.642456 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:43.642463 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:43.642474 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:43.642984 (+ 510us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:43.643054 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:43.643074 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:43.643159 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:43.643502 (+ 343us) server_negotiation.cc:299] Negotiation successful
0809 04:48:43.643799 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":289,"threads_started":1}
I20260809 04:48:43.649544 2467 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e0a95bb9a894501a886932306e4a79b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e0a95bb9a894501a886932306e4a79b" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 39683 } }
I20260809 04:48:43.649893 2467 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e0a95bb9a894501a886932306e4a79b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:43.650000 2467 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e0a95bb9a894501a886932306e4a79b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0a95bb9a894501a886932306e4a79b, State: Initialized, Role: FOLLOWER
I20260809 04:48:43.650578 2467 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e0a95bb9a894501a886932306e4a79b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e0a95bb9a894501a886932306e4a79b" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 39683 } }
I20260809 04:48:43.650835 2467 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e0a95bb9a894501a886932306e4a79b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:48:43.650986 2467 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e0a95bb9a894501a886932306e4a79b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:48:43.651165 2467 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e0a95bb9a894501a886932306e4a79b [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:43.653417 2467 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e0a95bb9a894501a886932306e4a79b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e0a95bb9a894501a886932306e4a79b" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 39683 } }
I20260809 04:48:43.653828 2467 leader_election.cc:302] T 00000000000000000000000000000000 P 5e0a95bb9a894501a886932306e4a79b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e0a95bb9a894501a886932306e4a79b; no voters:
I20260809 04:48:43.654332 2467 leader_election.cc:288] T 00000000000000000000000000000000 P 5e0a95bb9a894501a886932306e4a79b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:48:43.654475 2472 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e0a95bb9a894501a886932306e4a79b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:48:43.655222 2472 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e0a95bb9a894501a886932306e4a79b [term 1 LEADER]: Becoming Leader. State: Replica: 5e0a95bb9a894501a886932306e4a79b, State: Running, Role: LEADER
I20260809 04:48:43.655668 2472 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e0a95bb9a894501a886932306e4a79b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e0a95bb9a894501a886932306e4a79b" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 39683 } }
I20260809 04:48:43.656453 2467 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e0a95bb9a894501a886932306e4a79b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:48:43.658420 2474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e0a95bb9a894501a886932306e4a79b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e0a95bb9a894501a886932306e4a79b. Latest consensus state: current_term: 1 leader_uuid: "5e0a95bb9a894501a886932306e4a79b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e0a95bb9a894501a886932306e4a79b" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 39683 } } }
I20260809 04:48:43.658515 2473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e0a95bb9a894501a886932306e4a79b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e0a95bb9a894501a886932306e4a79b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e0a95bb9a894501a886932306e4a79b" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 39683 } } }
I20260809 04:48:43.658819 2474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e0a95bb9a894501a886932306e4a79b [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:43.658906 2473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e0a95bb9a894501a886932306e4a79b [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:43.664867 2477 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:48:43.669835 2477 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:48:43.678071 2477 catalog_manager.cc:1347] Generated new cluster ID: b1904e1c5fb4423e9158ac689b9abab3
I20260809 04:48:43.678215 2477 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:48:43.698227 2477 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:48:43.699867 2477 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:48:43.710807 2477 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e0a95bb9a894501a886932306e4a79b: Generated new TSK 0
I20260809 04:48:43.711644 2477 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:48:43.727454 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.1:0
--local_ip_for_outbound_sockets=127.12.234.1
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:39683
--builtin_ntp_servers=127.12.234.20:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 04:48:44.038757 2491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:44.039042 2491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:44.039184 2491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:44.045352 2491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:44.045423 2491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:44.045593 2491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.1
I20260809 04:48:44.053172 2491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.234.1
--webserver_port=0
--tserver_master_addrs=127.12.234.62:39683
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.1
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:44.054181 2491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:44.055635 2491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:44.067489 2496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:44.067457 2499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:44.067559 2497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:44.068140 2491 server_base.cc:1034] running on GCE node
I20260809 04:48:44.068610 2491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:44.069397 2491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:44.070592 2491 hybrid_clock.cc:648] HybridClock initialized: now 1786250924070559 us; error 31 us; skew 500 ppm
I20260809 04:48:44.070874 2491 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:48:44.072999 2491 webserver.cc:460] Webserver started at http://127.12.234.1:38637/ using document root <none> and password file <none>
I20260809 04:48:44.073598 2491 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:44.073710 2491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:44.073994 2491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:44.076520 2491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-0/data/instance:
uuid: "efb4d81e51e94ab09a362b3a8ac64648"
format_stamp: "Formatted at 2026-08-09 04:48:44 on dist-test-slave-1jjb"
server_key: "6ec299b373759ea00d6275137d768f04"
server_key_iv: "6d2bab93c8fcafa3769798346af0a57d"
server_key_version: "encryptionkey@0"
I20260809 04:48:44.077248 2491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance:
uuid: "efb4d81e51e94ab09a362b3a8ac64648"
format_stamp: "Formatted at 2026-08-09 04:48:44 on dist-test-slave-1jjb"
server_key: "6ec299b373759ea00d6275137d768f04"
server_key_iv: "6d2bab93c8fcafa3769798346af0a57d"
server_key_version: "encryptionkey@0"
I20260809 04:48:44.081734 2491 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:48:44.084681 2505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:44.085819 2491 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:48:44.085971 2491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-0/wal
uuid: "efb4d81e51e94ab09a362b3a8ac64648"
format_stamp: "Formatted at 2026-08-09 04:48:44 on dist-test-slave-1jjb"
server_key: "6ec299b373759ea00d6275137d768f04"
server_key_iv: "6d2bab93c8fcafa3769798346af0a57d"
server_key_version: "encryptionkey@0"
I20260809 04:48:44.086150 2491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:44.108407 2491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:44.109107 2491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:44.109398 2491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:44.110280 2491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:44.111459 2491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:48:44.111570 2491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:44.111680 2491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:44.111757 2491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:44.141695 2491 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.1:42385
I20260809 04:48:44.141760 2622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.1:42385 every 8 connection(s)
I20260809 04:48:44.142764 2491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-0/data/info.pb
I20260809 04:48:44.146278 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 2491
I20260809 04:48:44.146430 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-0/wal/instance
I20260809 04:48:44.146816 13224 external_mini_cluster.cc:1428] Setting key 44e8b399595fb48a27485f39575ca52e
I20260809 04:48:44.149310 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.2:0
--local_ip_for_outbound_sockets=127.12.234.2
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:39683
--builtin_ntp_servers=127.12.234.20:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 04:48:44.153847 2470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:44.145005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.1:39045 (local address 127.12.234.62:39683)
0809 04:48:44.145174 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:44.145178 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:44.146206 (+ 1028us) server_negotiation.cc:407] Connection header received
0809 04:48:44.147995 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:44.148000 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:44.148095 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:44.148178 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:44.149164 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:44.149973 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:44.151375 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:44.151729 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:44.151854 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:44.153072 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:44.153079 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:44.153087 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:44.153102 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:44.153149 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:44.153160 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:44.153250 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:44.153462 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 04:48:44.153553 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:48:44.154731 2625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:44.145309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:39683 (local address 127.12.234.1:39045)
0809 04:48:44.146059 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:44.146090 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:44.147755 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:44.148397 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:44.148405 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:44.148487 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:48:44.148993 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:44.149009 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:44.150159 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:44.150164 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:44.151214 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:44.151226 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:44.151938 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:44.151945 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:44.152185 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:44.152798 (+ 613us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:48:44.152800 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:48:44.152834 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:48:44.152898 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:48:44.153251 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:48:44.153264 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:48:44.153273 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 04:48:44.153444 (+ 171us) client_negotiation.cc:240] Negotiation successful
0809 04:48:44.153767 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":251,"threads_started":1}
I20260809 04:48:44.156059 2623 heartbeater.cc:347] Connected to a master server at 127.12.234.62:39683
I20260809 04:48:44.156381 2623 heartbeater.cc:464] Registering TS with master...
I20260809 04:48:44.157130 2623 heartbeater.cc:511] Master 127.12.234.62:39683 requested a full tablet report, sending...
I20260809 04:48:44.159241 2432 ts_manager.cc:194] Registered new tserver with Master: efb4d81e51e94ab09a362b3a8ac64648 (127.12.234.1:42385)
I20260809 04:48:44.161188 2432 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.234.1:39045
W20260809 04:48:44.462365 2626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:44.462690 2626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:44.462790 2626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:44.468954 2626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:44.469035 2626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:44.469141 2626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.2
I20260809 04:48:44.476729 2626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.234.2
--webserver_port=0
--tserver_master_addrs=127.12.234.62:39683
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.2
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:44.477625 2626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:44.479127 2626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:44.491056 2631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:44.491091 2634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:44.491127 2632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:44.491765 2626 server_base.cc:1034] running on GCE node
I20260809 04:48:44.492390 2626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:44.493244 2626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:44.494427 2626 hybrid_clock.cc:648] HybridClock initialized: now 1786250924494386 us; error 36 us; skew 500 ppm
I20260809 04:48:44.494720 2626 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:48:44.497005 2626 webserver.cc:460] Webserver started at http://127.12.234.2:39003/ using document root <none> and password file <none>
I20260809 04:48:44.497762 2626 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:44.497915 2626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:44.498291 2626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:44.501410 2626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-1/data/instance:
uuid: "964c184aa07249349ba99dd62769ec7d"
format_stamp: "Formatted at 2026-08-09 04:48:44 on dist-test-slave-1jjb"
server_key: "167af437013df9abffe6ca50e51dbe0c"
server_key_iv: "c73847ad4d34cbc08a7ac3ee49c855e6"
server_key_version: "encryptionkey@0"
I20260809 04:48:44.502063 2626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance:
uuid: "964c184aa07249349ba99dd62769ec7d"
format_stamp: "Formatted at 2026-08-09 04:48:44 on dist-test-slave-1jjb"
server_key: "167af437013df9abffe6ca50e51dbe0c"
server_key_iv: "c73847ad4d34cbc08a7ac3ee49c855e6"
server_key_version: "encryptionkey@0"
I20260809 04:48:44.506685 2626 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 04:48:44.509734 2640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:44.510969 2626 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260809 04:48:44.511133 2626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-1/wal
uuid: "964c184aa07249349ba99dd62769ec7d"
format_stamp: "Formatted at 2026-08-09 04:48:44 on dist-test-slave-1jjb"
server_key: "167af437013df9abffe6ca50e51dbe0c"
server_key_iv: "c73847ad4d34cbc08a7ac3ee49c855e6"
server_key_version: "encryptionkey@0"
I20260809 04:48:44.511339 2626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:44.529815 2626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:44.530544 2626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:44.530813 2626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:44.531641 2626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:44.533027 2626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:48:44.533125 2626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:44.533291 2626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:44.533376 2626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:44.563158 2626 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.2:44015
I20260809 04:48:44.563228 2764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.2:44015 every 8 connection(s)
I20260809 04:48:44.564277 2626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-1/data/info.pb
I20260809 04:48:44.568540 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 2626
I20260809 04:48:44.568725 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-1/wal/instance
I20260809 04:48:44.569161 13224 external_mini_cluster.cc:1428] Setting key 3c50de1d2b17d381d5cce07acf379426
I20260809 04:48:44.572165 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.234.3:0
--local_ip_for_outbound_sockets=127.12.234.3
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:39683
--builtin_ntp_servers=127.12.234.20:46853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 04:48:44.576421 2470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:44.566472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.2:46927 (local address 127.12.234.62:39683)
0809 04:48:44.566662 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:44.566667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:44.567907 (+ 1240us) server_negotiation.cc:407] Connection header received
0809 04:48:44.569809 (+ 1902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:44.569813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:44.569907 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:44.570014 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:44.571339 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:44.572336 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:44.573670 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:44.574272 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:44.574419 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:44.575565 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:44.575572 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:44.575578 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:44.575601 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:44.575651 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:44.575666 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:44.575745 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:44.575949 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 04:48:44.576044 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 04:48:44.577174 2767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:44.566996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:39683 (local address 127.12.234.2:46927)
0809 04:48:44.567756 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:44.567785 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:44.569529 (+ 1744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:44.570273 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:44.570283 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:44.570388 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:48:44.571161 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:44.571186 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:44.572456 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:44.572460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:44.573535 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:44.573545 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:44.574473 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:44.574476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:44.574638 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:44.575266 (+ 628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:48:44.575269 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:48:44.575302 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:48:44.575390 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:48:44.575752 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:48:44.575759 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:48:44.575764 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 04:48:44.575901 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 04:48:44.576203 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":248,"threads_started":1}
I20260809 04:48:44.578337 2765 heartbeater.cc:347] Connected to a master server at 127.12.234.62:39683
I20260809 04:48:44.578617 2765 heartbeater.cc:464] Registering TS with master...
I20260809 04:48:44.579418 2765 heartbeater.cc:511] Master 127.12.234.62:39683 requested a full tablet report, sending...
I20260809 04:48:44.580855 2432 ts_manager.cc:194] Registered new tserver with Master: 964c184aa07249349ba99dd62769ec7d (127.12.234.2:44015)
I20260809 04:48:44.581535 2432 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.234.2:46927
W20260809 04:48:44.873994 2768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:44.874225 2768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:44.874369 2768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:44.880810 2768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:44.880892 2768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:44.881121 2768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.234.3
I20260809 04:48:44.888896 2768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:46853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.234.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.234.3
--webserver_port=0
--tserver_master_addrs=127.12.234.62:39683
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.234.3
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:44.890113 2768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:44.891844 2768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:44.903851 2773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:44.903898 2774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:44.903925 2776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:44.905001 2768 server_base.cc:1034] running on GCE node
I20260809 04:48:44.905463 2768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:44.906190 2768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:44.907375 2768 hybrid_clock.cc:648] HybridClock initialized: now 1786250924907334 us; error 34 us; skew 500 ppm
I20260809 04:48:44.907660 2768 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 04:48:44.909866 2768 webserver.cc:460] Webserver started at http://127.12.234.3:39367/ using document root <none> and password file <none>
I20260809 04:48:44.910504 2768 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:44.910609 2768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:44.910895 2768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:44.913305 2768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-2/data/instance:
uuid: "48902464c0cd409c9f09d7473d4db611"
format_stamp: "Formatted at 2026-08-09 04:48:44 on dist-test-slave-1jjb"
server_key: "a59e8ac93587fc419553bb482cfe0419"
server_key_iv: "799d9366140e7ffbed9acb7f9ce32af9"
server_key_version: "encryptionkey@0"
I20260809 04:48:44.914115 2768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance:
uuid: "48902464c0cd409c9f09d7473d4db611"
format_stamp: "Formatted at 2026-08-09 04:48:44 on dist-test-slave-1jjb"
server_key: "a59e8ac93587fc419553bb482cfe0419"
server_key_iv: "799d9366140e7ffbed9acb7f9ce32af9"
server_key_version: "encryptionkey@0"
I20260809 04:48:44.918641 2768 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 04:48:44.921708 2783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:44.922811 2768 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 04:48:44.922971 2768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-2/wal
uuid: "48902464c0cd409c9f09d7473d4db611"
format_stamp: "Formatted at 2026-08-09 04:48:44 on dist-test-slave-1jjb"
server_key: "a59e8ac93587fc419553bb482cfe0419"
server_key_iv: "799d9366140e7ffbed9acb7f9ce32af9"
server_key_version: "encryptionkey@0"
I20260809 04:48:44.923168 2768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:44.943934 2768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:44.944562 2768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:44.944823 2768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:44.945601 2768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 04:48:44.946779 2768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 04:48:44.946892 2768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:44.947000 2768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 04:48:44.947085 2768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:44.976648 2768 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.3:39291
I20260809 04:48:44.976738 2910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.3:39291 every 8 connection(s)
I20260809 04:48:44.977671 2768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-2/data/info.pb
I20260809 04:48:44.981050 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 2768
I20260809 04:48:44.981217 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-2/wal/instance
I20260809 04:48:44.981709 13224 external_mini_cluster.cc:1428] Setting key 8fb4a0e31fadd66bbf79916206d42e33
I20260809 04:48:44.989658 2470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:44.979791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:51633 (local address 127.12.234.62:39683)
0809 04:48:44.979966 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:44.979971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:44.981341 (+ 1370us) server_negotiation.cc:407] Connection header received
0809 04:48:44.983824 (+ 2483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:44.983833 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:44.983931 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:44.984046 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:44.985158 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:44.985909 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:44.987187 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:44.987587 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:44.987687 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:44.988843 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:44.988847 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:44.988851 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:44.988912 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:44.988974 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:44.988989 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:44.989083 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:44.989361 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 04:48:44.989437 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 04:48:44.990566 2913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:44.980284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:39683 (local address 127.12.234.3:51633)
0809 04:48:44.981189 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:44.981232 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:44.983371 (+ 2139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:44.984281 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:44.984289 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:44.984371 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:48:44.985002 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:44.985019 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:44.986036 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:44.986040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:44.987038 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:44.987049 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:44.987761 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:44.987765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:44.987929 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:44.988577 (+ 648us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 04:48:44.988579 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:48:44.988610 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 04:48:44.988671 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:48:44.989121 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:48:44.989129 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:48:44.989134 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 04:48:44.989266 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 04:48:44.989518 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":239,"threads_started":1}
I20260809 04:48:44.991751 2911 heartbeater.cc:347] Connected to a master server at 127.12.234.62:39683
I20260809 04:48:44.992053 2911 heartbeater.cc:464] Registering TS with master...
I20260809 04:48:44.992717 2911 heartbeater.cc:511] Master 127.12.234.62:39683 requested a full tablet report, sending...
I20260809 04:48:44.994009 2432 ts_manager.cc:194] Registered new tserver with Master: 48902464c0cd409c9f09d7473d4db611 (127.12.234.3:39291)
I20260809 04:48:44.994695 2432 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.234.3:51633
I20260809 04:48:44.996834 13224 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 04:48:45.006036 2470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:45.000707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51622 (local address 127.12.234.62:39683)
0809 04:48:45.000870 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:45.000876 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:45.001484 (+ 608us) server_negotiation.cc:407] Connection header received
0809 04:48:45.001655 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:45.001659 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:45.001721 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:45.001822 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:45.002206 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.002840 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.004153 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.004509 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.004602 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:45.005427 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:45.005431 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:45.005434 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:45.005447 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:45.005473 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:45.005483 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:45.005565 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:45.005672 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 04:48:45.005733 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 04:48:45.009087 2432 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51622:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 04:48:45.014999 2432 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 04:48:45.042938 2924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:45.024741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:44015 (local address 127.0.0.1:35536)
0809 04:48:45.031586 (+ 6845us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:45.031629 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:45.031821 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:45.036596 (+ 4775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:45.036602 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:45.036626 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:45.037024 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:45.037037 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.038638 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.038645 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:45.040966 (+ 2321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:45.040977 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.042097 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.042102 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:45.042321 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:45.042382 (+ 61us) client_negotiation.cc:769] Sending connection context
0809 04:48:45.042501 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 04:48:45.042664 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6555,"thread_start_us":6406,"threads_started":1}
I20260809 04:48:45.043278 2926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:45.024844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35536 (local address 127.12.234.2:44015)
0809 04:48:45.035694 (+ 10850us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:45.035699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:45.035721 (+ 22us) server_negotiation.cc:407] Connection header received
0809 04:48:45.035795 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:45.035799 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:45.035941 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:45.036164 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:45.037207 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.038498 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.041141 (+ 2643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.041914 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.042040 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:45.042297 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:45.042569 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 04:48:45.042718 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10734,"thread_start_us":10562,"threads_started":1}
I20260809 04:48:45.044541 2925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:45.024816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:42385 (local address 127.0.0.1:40218)
0809 04:48:45.037294 (+ 12478us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:45.037325 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:45.037499 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:45.038231 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:45.038235 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:45.038257 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:45.038561 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:45.038575 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.040353 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.040357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:45.042192 (+ 1835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:45.042207 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.043814 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.043820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:45.043979 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:45.044011 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 04:48:45.044137 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 04:48:45.044240 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12341,"mutex_wait_us":20,"thread_start_us":12779,"threads_started":1}
I20260809 04:48:45.045318 2928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:45.025456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40218 (local address 127.12.234.1:42385)
0809 04:48:45.035415 (+ 9959us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:45.035425 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:45.037474 (+ 2049us) server_negotiation.cc:407] Connection header received
0809 04:48:45.037662 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:45.037669 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:45.037852 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:45.038003 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:45.038727 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.040202 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.042392 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.043646 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.043768 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:45.043985 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:45.044846 (+ 861us) server_negotiation.cc:299] Negotiation successful
0809 04:48:45.045007 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9769,"thread_start_us":9632,"threads_started":1}
I20260809 04:48:45.045701 2927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:45.025398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.3:39291 (local address 127.0.0.1:48668)
0809 04:48:45.038104 (+ 12706us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:45.038115 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:45.038231 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:45.038674 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:45.038679 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:45.038708 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:45.039167 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:45.039180 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.040953 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.040957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:45.042801 (+ 1844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:45.042816 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.045049 (+ 2233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.045054 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:45.045199 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:45.045226 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 04:48:45.045302 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 04:48:45.045399 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12621,"mutex_wait_us":22,"thread_start_us":12511,"threads_started":1}
I20260809 04:48:45.045759 2929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:45.025997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48668 (local address 127.12.234.3:39291)
0809 04:48:45.034527 (+ 8530us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:45.034538 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:45.038204 (+ 3666us) server_negotiation.cc:407] Connection header received
0809 04:48:45.038387 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:45.038391 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:45.038519 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:45.038691 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:45.039387 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.040807 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.043014 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.044903 (+ 1889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.045047 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:45.045267 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:45.045332 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 04:48:45.045461 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8322,"thread_start_us":8185,"threads_started":1}
I20260809 04:48:45.046725 2692 tablet_service.cc:1467] Processing CreateTablet for tablet c17e875698d84211aa724bd14df588a4 (DEFAULT_TABLE table=test-table [id=38661239cce44e01851839a7360e8a8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:48:45.048188 2692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c17e875698d84211aa724bd14df588a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:45.048236 2557 tablet_service.cc:1467] Processing CreateTablet for tablet c17e875698d84211aa724bd14df588a4 (DEFAULT_TABLE table=test-table [id=38661239cce44e01851839a7360e8a8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:48:45.049424 2557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c17e875698d84211aa724bd14df588a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:45.049130 2840 tablet_service.cc:1467] Processing CreateTablet for tablet c17e875698d84211aa724bd14df588a4 (DEFAULT_TABLE table=test-table [id=38661239cce44e01851839a7360e8a8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 04:48:45.050417 2840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c17e875698d84211aa724bd14df588a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:45.059557 2930 tablet_bootstrap.cc:492] T c17e875698d84211aa724bd14df588a4 P efb4d81e51e94ab09a362b3a8ac64648: Bootstrap starting.
I20260809 04:48:45.061625 2932 tablet_bootstrap.cc:492] T c17e875698d84211aa724bd14df588a4 P 964c184aa07249349ba99dd62769ec7d: Bootstrap starting.
I20260809 04:48:45.062852 2930 tablet_bootstrap.cc:654] T c17e875698d84211aa724bd14df588a4 P efb4d81e51e94ab09a362b3a8ac64648: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:45.062994 2933 tablet_bootstrap.cc:492] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611: Bootstrap starting.
I20260809 04:48:45.064406 2930 log.cc:824] T c17e875698d84211aa724bd14df588a4 P efb4d81e51e94ab09a362b3a8ac64648: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:45.064728 2932 tablet_bootstrap.cc:654] T c17e875698d84211aa724bd14df588a4 P 964c184aa07249349ba99dd62769ec7d: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:45.065923 2933 tablet_bootstrap.cc:654] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:45.066057 2932 log.cc:824] T c17e875698d84211aa724bd14df588a4 P 964c184aa07249349ba99dd62769ec7d: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:45.066920 2933 log.cc:824] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:45.069311 2932 tablet_bootstrap.cc:492] T c17e875698d84211aa724bd14df588a4 P 964c184aa07249349ba99dd62769ec7d: No bootstrap required, opened a new log
I20260809 04:48:45.069633 2932 ts_tablet_manager.cc:1397] T c17e875698d84211aa724bd14df588a4 P 964c184aa07249349ba99dd62769ec7d: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 04:48:45.070081 2933 tablet_bootstrap.cc:492] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611: No bootstrap required, opened a new log
I20260809 04:48:45.070331 2933 ts_tablet_manager.cc:1397] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260809 04:48:45.074406 2930 tablet_bootstrap.cc:492] T c17e875698d84211aa724bd14df588a4 P efb4d81e51e94ab09a362b3a8ac64648: No bootstrap required, opened a new log
I20260809 04:48:45.074661 2930 ts_tablet_manager.cc:1397] T c17e875698d84211aa724bd14df588a4 P efb4d81e51e94ab09a362b3a8ac64648: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.013s
I20260809 04:48:45.077286 2933 raft_consensus.cc:348] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964c184aa07249349ba99dd62769ec7d" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 44015 } } peers { permanent_uuid: "efb4d81e51e94ab09a362b3a8ac64648" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 42385 } } peers { permanent_uuid: "48902464c0cd409c9f09d7473d4db611" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 39291 } }
I20260809 04:48:45.077605 2933 raft_consensus.cc:374] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:45.077721 2933 raft_consensus.cc:729] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48902464c0cd409c9f09d7473d4db611, State: Initialized, Role: FOLLOWER
I20260809 04:48:45.078279 2933 consensus_queue.cc:260] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964c184aa07249349ba99dd62769ec7d" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 44015 } } peers { permanent_uuid: "efb4d81e51e94ab09a362b3a8ac64648" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 42385 } } peers { permanent_uuid: "48902464c0cd409c9f09d7473d4db611" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 39291 } }
I20260809 04:48:45.079308 2911 heartbeater.cc:503] Master 127.12.234.62:39683 was elected leader, sending a full tablet report...
I20260809 04:48:45.079823 2933 ts_tablet_manager.cc:1428] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611: Time spent starting tablet: real 0.009s user 0.008s sys 0.002s
I20260809 04:48:45.079865 2932 raft_consensus.cc:348] T c17e875698d84211aa724bd14df588a4 P 964c184aa07249349ba99dd62769ec7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964c184aa07249349ba99dd62769ec7d" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 44015 } } peers { permanent_uuid: "efb4d81e51e94ab09a362b3a8ac64648" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 42385 } } peers { permanent_uuid: "48902464c0cd409c9f09d7473d4db611" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 39291 } }
I20260809 04:48:45.080313 2932 raft_consensus.cc:374] T c17e875698d84211aa724bd14df588a4 P 964c184aa07249349ba99dd62769ec7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:45.080407 2932 raft_consensus.cc:729] T c17e875698d84211aa724bd14df588a4 P 964c184aa07249349ba99dd62769ec7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 964c184aa07249349ba99dd62769ec7d, State: Initialized, Role: FOLLOWER
I20260809 04:48:45.080971 2932 consensus_queue.cc:260] T c17e875698d84211aa724bd14df588a4 P 964c184aa07249349ba99dd62769ec7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964c184aa07249349ba99dd62769ec7d" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 44015 } } peers { permanent_uuid: "efb4d81e51e94ab09a362b3a8ac64648" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 42385 } } peers { permanent_uuid: "48902464c0cd409c9f09d7473d4db611" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 39291 } }
I20260809 04:48:45.084102 2765 heartbeater.cc:503] Master 127.12.234.62:39683 was elected leader, sending a full tablet report...
I20260809 04:48:45.083896 2930 raft_consensus.cc:348] T c17e875698d84211aa724bd14df588a4 P efb4d81e51e94ab09a362b3a8ac64648 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964c184aa07249349ba99dd62769ec7d" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 44015 } } peers { permanent_uuid: "efb4d81e51e94ab09a362b3a8ac64648" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 42385 } } peers { permanent_uuid: "48902464c0cd409c9f09d7473d4db611" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 39291 } }
I20260809 04:48:45.084295 2930 raft_consensus.cc:374] T c17e875698d84211aa724bd14df588a4 P efb4d81e51e94ab09a362b3a8ac64648 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:45.084403 2930 raft_consensus.cc:729] T c17e875698d84211aa724bd14df588a4 P efb4d81e51e94ab09a362b3a8ac64648 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efb4d81e51e94ab09a362b3a8ac64648, State: Initialized, Role: FOLLOWER
I20260809 04:48:45.084858 2932 ts_tablet_manager.cc:1428] T c17e875698d84211aa724bd14df588a4 P 964c184aa07249349ba99dd62769ec7d: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260809 04:48:45.084923 2930 consensus_queue.cc:260] T c17e875698d84211aa724bd14df588a4 P efb4d81e51e94ab09a362b3a8ac64648 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964c184aa07249349ba99dd62769ec7d" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 44015 } } peers { permanent_uuid: "efb4d81e51e94ab09a362b3a8ac64648" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 42385 } } peers { permanent_uuid: "48902464c0cd409c9f09d7473d4db611" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 39291 } }
I20260809 04:48:45.085933 2623 heartbeater.cc:503] Master 127.12.234.62:39683 was elected leader, sending a full tablet report...
I20260809 04:48:45.086566 2930 ts_tablet_manager.cc:1428] T c17e875698d84211aa724bd14df588a4 P efb4d81e51e94ab09a362b3a8ac64648: Time spent starting tablet: real 0.012s user 0.009s sys 0.002s
W20260809 04:48:45.144888 2624 tablet.cc:2369] T c17e875698d84211aa724bd14df588a4 P efb4d81e51e94ab09a362b3a8ac64648: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 04:48:45.229192 2912 tablet.cc:2369] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:48:45.253067 2939 raft_consensus.cc:482] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 04:48:45.253340 2939 raft_consensus.cc:504] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964c184aa07249349ba99dd62769ec7d" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 44015 } } peers { permanent_uuid: "efb4d81e51e94ab09a362b3a8ac64648" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 42385 } } peers { permanent_uuid: "48902464c0cd409c9f09d7473d4db611" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 39291 } }
I20260809 04:48:45.254809 2939 leader_election.cc:288] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 964c184aa07249349ba99dd62769ec7d (127.12.234.2:44015), efb4d81e51e94ab09a362b3a8ac64648 (127.12.234.1:42385)
I20260809 04:48:45.260851 2928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:45.255223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:50035 (local address 127.12.234.1:42385)
0809 04:48:45.255418 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:45.255424 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:45.255608 (+ 184us) server_negotiation.cc:407] Connection header received
0809 04:48:45.255793 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:45.255798 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:45.255880 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:45.255996 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:45.256568 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.257463 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.259089 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.259869 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.260065 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:45.260185 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:45.260448 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 04:48:45.260575 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 04:48:45.260855 2913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:45.255287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.1:42385 (local address 127.12.234.3:50035)
0809 04:48:45.255474 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:45.255495 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:45.255649 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:45.256032 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:45.256037 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:45.256117 (+ 80us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:45.256394 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:45.256409 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.257578 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.257582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:45.258881 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:45.258893 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.260039 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.260046 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:45.260281 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:45.260311 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 04:48:45.260420 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 04:48:45.260557 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 04:48:45.261276 2926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:45.255251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.234.3:37991 (local address 127.12.234.2:44015)
0809 04:48:45.255474 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:45.255483 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:45.256341 (+ 858us) server_negotiation.cc:407] Connection header received
0809 04:48:45.256453 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:45.256461 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:45.256552 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:45.256707 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 04:48:45.257316 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.258112 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.259658 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.260468 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.260634 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:45.260693 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:45.260933 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 04:48:45.261067 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 04:48:45.261585 2942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:45.255287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.2:44015 (local address 127.12.234.3:37991)
0809 04:48:45.256240 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0809 04:48:45.256254 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 04:48:45.256351 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:48:45.256734 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:48:45.256739 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:48:45.256764 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 04:48:45.257114 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:45.257124 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.258253 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.258257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:45.259483 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:48:45.259494 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.260633 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.260639 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:48:45.260790 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:45.260809 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 04:48:45.261180 (+ 371us) client_negotiation.cc:240] Negotiation successful
0809 04:48:45.261299 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":819,"mutex_wait_us":160,"thread_start_us":431,"threads_started":1}
I20260809 04:48:45.261904 2578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17e875698d84211aa724bd14df588a4" candidate_uuid: "48902464c0cd409c9f09d7473d4db611" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efb4d81e51e94ab09a362b3a8ac64648" is_pre_election: true
I20260809 04:48:45.262537 2578 raft_consensus.cc:2454] T c17e875698d84211aa724bd14df588a4 P efb4d81e51e94ab09a362b3a8ac64648 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48902464c0cd409c9f09d7473d4db611 in term 0.
I20260809 04:48:45.262543 2715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17e875698d84211aa724bd14df588a4" candidate_uuid: "48902464c0cd409c9f09d7473d4db611" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "964c184aa07249349ba99dd62769ec7d" is_pre_election: true
I20260809 04:48:45.263077 2715 raft_consensus.cc:2454] T c17e875698d84211aa724bd14df588a4 P 964c184aa07249349ba99dd62769ec7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48902464c0cd409c9f09d7473d4db611 in term 0.
I20260809 04:48:45.263290 2785 leader_election.cc:302] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48902464c0cd409c9f09d7473d4db611, efb4d81e51e94ab09a362b3a8ac64648; no voters:
I20260809 04:48:45.263705 2939 raft_consensus.cc:2781] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 04:48:45.263829 2939 raft_consensus.cc:482] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 04:48:45.263960 2939 raft_consensus.cc:3037] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:45.265974 2939 raft_consensus.cc:504] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964c184aa07249349ba99dd62769ec7d" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 44015 } } peers { permanent_uuid: "efb4d81e51e94ab09a362b3a8ac64648" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 42385 } } peers { permanent_uuid: "48902464c0cd409c9f09d7473d4db611" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 39291 } }
I20260809 04:48:45.266610 2939 leader_election.cc:288] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611 [CANDIDATE]: Term 1 election: Requested vote from peers 964c184aa07249349ba99dd62769ec7d (127.12.234.2:44015), efb4d81e51e94ab09a362b3a8ac64648 (127.12.234.1:42385)
I20260809 04:48:45.266950 2715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17e875698d84211aa724bd14df588a4" candidate_uuid: "48902464c0cd409c9f09d7473d4db611" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "964c184aa07249349ba99dd62769ec7d"
I20260809 04:48:45.267124 2715 raft_consensus.cc:3037] T c17e875698d84211aa724bd14df588a4 P 964c184aa07249349ba99dd62769ec7d [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:45.267171 2578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c17e875698d84211aa724bd14df588a4" candidate_uuid: "48902464c0cd409c9f09d7473d4db611" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efb4d81e51e94ab09a362b3a8ac64648"
I20260809 04:48:45.267371 2578 raft_consensus.cc:3037] T c17e875698d84211aa724bd14df588a4 P efb4d81e51e94ab09a362b3a8ac64648 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:45.269471 2715 raft_consensus.cc:2454] T c17e875698d84211aa724bd14df588a4 P 964c184aa07249349ba99dd62769ec7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48902464c0cd409c9f09d7473d4db611 in term 1.
I20260809 04:48:45.269852 2787 leader_election.cc:302] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48902464c0cd409c9f09d7473d4db611, 964c184aa07249349ba99dd62769ec7d; no voters:
I20260809 04:48:45.269903 2578 raft_consensus.cc:2454] T c17e875698d84211aa724bd14df588a4 P efb4d81e51e94ab09a362b3a8ac64648 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48902464c0cd409c9f09d7473d4db611 in term 1.
I20260809 04:48:45.270129 2939 raft_consensus.cc:2781] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:48:45.270653 2939 raft_consensus.cc:686] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611 [term 1 LEADER]: Becoming Leader. State: Replica: 48902464c0cd409c9f09d7473d4db611, State: Running, Role: LEADER
I20260809 04:48:45.271181 2939 consensus_queue.cc:237] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964c184aa07249349ba99dd62769ec7d" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 44015 } } peers { permanent_uuid: "efb4d81e51e94ab09a362b3a8ac64648" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 42385 } } peers { permanent_uuid: "48902464c0cd409c9f09d7473d4db611" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 39291 } }
I20260809 04:48:45.275637 2432 catalog_manager.cc:5495] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48902464c0cd409c9f09d7473d4db611 (127.12.234.3). New cstate: current_term: 1 leader_uuid: "48902464c0cd409c9f09d7473d4db611" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "964c184aa07249349ba99dd62769ec7d" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 44015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efb4d81e51e94ab09a362b3a8ac64648" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 42385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48902464c0cd409c9f09d7473d4db611" member_type: VOTER last_known_addr { host: "127.12.234.3" port: 39291 } health_report { overall_health: HEALTHY } } }
W20260809 04:48:45.316040 2766 tablet.cc:2369] T c17e875698d84211aa724bd14df588a4 P 964c184aa07249349ba99dd62769ec7d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:48:45.335448 2929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:45.330546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48670 (local address 127.12.234.3:39291)
0809 04:48:45.330732 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:45.330738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:45.330849 (+ 111us) server_negotiation.cc:407] Connection header received
0809 04:48:45.331062 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:45.331065 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:45.331130 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:45.331264 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:48:45.331585 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.332262 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.333533 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.333883 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.334003 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:45.334470 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:48:45.334916 (+ 446us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:48:45.335059 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:45.335153 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 04:48:45.335224 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 04:48:45.342517 2715 raft_consensus.cc:1261] T c17e875698d84211aa724bd14df588a4 P 964c184aa07249349ba99dd62769ec7d [term 1 FOLLOWER]: Refusing update from remote peer 48902464c0cd409c9f09d7473d4db611: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:48:45.342581 2578 raft_consensus.cc:1261] T c17e875698d84211aa724bd14df588a4 P efb4d81e51e94ab09a362b3a8ac64648 [term 1 FOLLOWER]: Refusing update from remote peer 48902464c0cd409c9f09d7473d4db611: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:48:45.343362 2943 consensus_queue.cc:1035] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611 [LEADER]: Connected to new peer: Peer: permanent_uuid: "964c184aa07249349ba99dd62769ec7d" member_type: VOTER last_known_addr { host: "127.12.234.2" port: 44015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:48:45.343716 2939 consensus_queue.cc:1035] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efb4d81e51e94ab09a362b3a8ac64648" member_type: VOTER last_known_addr { host: "127.12.234.1" port: 42385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:48:45.353744 2947 mvcc.cc:205] Tried to move back new op lower bound from 7316483790194446336 to 7316483789915033600. Current Snapshot: MvccSnapshot[applied={T|T < 7316483790194446336}]
I20260809 04:48:45.366560 2431 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:48:45.366811 2431 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 04:48:45.370453 2431 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5e0a95bb9a894501a886932306e4a79b: Sending DeleteTablet for 3 replicas of tablet c17e875698d84211aa724bd14df588a4
I20260809 04:48:45.371574 2692 tablet_service.cc:1514] Processing DeleteTablet for tablet c17e875698d84211aa724bd14df588a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:48:45 UTC) from {username='slave'} at 127.0.0.1:35536
I20260809 04:48:45.371737 2557 tablet_service.cc:1514] Processing DeleteTablet for tablet c17e875698d84211aa724bd14df588a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:48:45 UTC) from {username='slave'} at 127.0.0.1:40218
I20260809 04:48:45.371941 2840 tablet_service.cc:1514] Processing DeleteTablet for tablet c17e875698d84211aa724bd14df588a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 04:48:45 UTC) from {username='slave'} at 127.0.0.1:48668
I20260809 04:48:45.373103 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 2491
I20260809 04:48:45.381280 2958 tablet_replica.cc:330] T c17e875698d84211aa724bd14df588a4 P efb4d81e51e94ab09a362b3a8ac64648: stopping tablet replica
I20260809 04:48:45.381716 2958 raft_consensus.cc:2229] T c17e875698d84211aa724bd14df588a4 P efb4d81e51e94ab09a362b3a8ac64648 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:48:45.382208 2958 raft_consensus.cc:2258] T c17e875698d84211aa724bd14df588a4 P efb4d81e51e94ab09a362b3a8ac64648 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:48:45.384558 2958 ts_tablet_manager.cc:1905] T c17e875698d84211aa724bd14df588a4 P efb4d81e51e94ab09a362b3a8ac64648: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:48:45.391799 2958 ts_tablet_manager.cc:1918] T c17e875698d84211aa724bd14df588a4 P efb4d81e51e94ab09a362b3a8ac64648: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:48:45.392016 2958 log.cc:1196] T c17e875698d84211aa724bd14df588a4 P efb4d81e51e94ab09a362b3a8ac64648: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-0/wal/wals/c17e875698d84211aa724bd14df588a4
I20260809 04:48:45.392755 2958 ts_tablet_manager.cc:1939] T c17e875698d84211aa724bd14df588a4 P efb4d81e51e94ab09a362b3a8ac64648: Deleting consensus metadata
I20260809 04:48:45.393433 2957 tablet_replica.cc:330] T c17e875698d84211aa724bd14df588a4 P 964c184aa07249349ba99dd62769ec7d: stopping tablet replica
I20260809 04:48:45.393953 2957 raft_consensus.cc:2229] T c17e875698d84211aa724bd14df588a4 P 964c184aa07249349ba99dd62769ec7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 04:48:45.394515 2957 raft_consensus.cc:2258] T c17e875698d84211aa724bd14df588a4 P 964c184aa07249349ba99dd62769ec7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:48:45.394698 2415 catalog_manager.cc:4841] TS efb4d81e51e94ab09a362b3a8ac64648 (127.12.234.1:42385): tablet c17e875698d84211aa724bd14df588a4 (table test-table [id=38661239cce44e01851839a7360e8a8c]) successfully deleted
I20260809 04:48:45.396440 2959 tablet_replica.cc:330] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611: stopping tablet replica
I20260809 04:48:45.396606 2957 ts_tablet_manager.cc:1905] T c17e875698d84211aa724bd14df588a4 P 964c184aa07249349ba99dd62769ec7d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:48:45.396790 2959 raft_consensus.cc:2229] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611 [term 1 LEADER]: Raft consensus shutting down.
I20260809 04:48:45.397511 2959 raft_consensus.cc:2258] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 04:48:45.399319 2928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:45.374904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40228 (local address 127.12.234.1:42385)
0809 04:48:45.375120 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:45.375129 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:45.381071 (+ 5942us) server_negotiation.cc:407] Connection header received
0809 04:48:45.382079 (+ 1008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:45.382089 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:45.382199 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:45.382279 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:45.383704 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.385180 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.389512 (+ 4332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.390572 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.390719 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:45.397661 (+ 6942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:45.397672 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:45.397687 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:45.398331 (+ 644us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:45.398396 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:45.398420 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:45.398533 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:45.398756 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 04:48:45.398894 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 04:48:45.400481 2618 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:45.401538 2959 ts_tablet_manager.cc:1905] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 04:48:45.409049 2959 ts_tablet_manager.cc:1918] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:48:45.409242 2959 log.cc:1196] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-2/wal/wals/c17e875698d84211aa724bd14df588a4
I20260809 04:48:45.409890 2959 ts_tablet_manager.cc:1939] T c17e875698d84211aa724bd14df588a4 P 48902464c0cd409c9f09d7473d4db611: Deleting consensus metadata
I20260809 04:48:45.410022 2957 ts_tablet_manager.cc:1918] T c17e875698d84211aa724bd14df588a4 P 964c184aa07249349ba99dd62769ec7d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 04:48:45.410175 2957 log.cc:1196] T c17e875698d84211aa724bd14df588a4 P 964c184aa07249349ba99dd62769ec7d: Deleting WAL directory at /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786250603326224-13224-0/minicluster-data/ts-1/wal/wals/c17e875698d84211aa724bd14df588a4
I20260809 04:48:45.410661 2957 ts_tablet_manager.cc:1939] T c17e875698d84211aa724bd14df588a4 P 964c184aa07249349ba99dd62769ec7d: Deleting consensus metadata
I20260809 04:48:45.411522 2417 catalog_manager.cc:4841] TS 48902464c0cd409c9f09d7473d4db611 (127.12.234.3:39291): tablet c17e875698d84211aa724bd14df588a4 (table test-table [id=38661239cce44e01851839a7360e8a8c]) successfully deleted
I20260809 04:48:45.412161 2418 catalog_manager.cc:4841] TS 964c184aa07249349ba99dd62769ec7d (127.12.234.2:44015): tablet c17e875698d84211aa724bd14df588a4 (table test-table [id=38661239cce44e01851839a7360e8a8c]) successfully deleted
I20260809 04:48:45.489758 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 2491
I20260809 04:48:45.510731 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 2626
I20260809 04:48:45.517098 2926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:45.511503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35544 (local address 127.12.234.2:44015)
0809 04:48:45.511673 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:45.511679 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:45.511697 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:48:45.511822 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:45.511826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:45.511934 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:45.512037 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:45.512424 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.513108 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.514330 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.514710 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.514828 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:45.515787 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:45.515796 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:45.515809 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:45.516458 (+ 649us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:45.516501 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:45.516520 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:45.516627 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:45.516723 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 04:48:45.516815 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 04:48:45.517683 2760 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:45.601639 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 2626
I20260809 04:48:45.622046 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 2768
I20260809 04:48:45.628731 2929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:45.622731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48676 (local address 127.12.234.3:39291)
0809 04:48:45.622903 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:45.622908 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:45.623044 (+ 136us) server_negotiation.cc:407] Connection header received
0809 04:48:45.623251 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:45.623258 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:45.623348 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:45.623466 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:45.623844 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.624597 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.625680 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:45.626233 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:45.626370 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:45.627413 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:45.627421 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:45.627433 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:45.627948 (+ 515us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:45.627988 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:45.628006 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:45.628228 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:45.628274 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 04:48:45.628370 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 04:48:45.629424 2906 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:45.735999 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 2768
I20260809 04:48:45.758044 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 2398
I20260809 04:48:45.759104 2462 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:45.856983 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 2398
2026-08-09T04:48:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2716 ms)
[----------] 6 tests from EncryptionPolicyTest (21594 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:48:45 dist-test-slave-1jjb krb5kdc[2970](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:48:45 dist-test-slave-1jjb krb5kdc[2970](info): set up 2 sockets
Aug 09 04:48:45 dist-test-slave-1jjb krb5kdc[2970](info): commencing operation
krb5kdc: starting...
W20260809 04:48:46.913676 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.004s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:48:47 dist-test-slave-1jjb krb5kdc[2970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:48:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:48:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:48:47.165683 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45177
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:48:47.468492 2983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:47.468736 2983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:47.468817 2983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:47.476897 2983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:48:47.477005 2983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:47.477070 2983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:47.477157 2983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:48:47.477214 2983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:48:47.487586 2983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45177
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:47.488659 2983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:47.489817 2983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:47.500478 2988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:47.500505 2992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:47.500624 2990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:47.501179 2983 server_base.cc:1034] running on GCE node
I20260809 04:48:47.501780 2983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:47.502876 2983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:47.504101 2983 hybrid_clock.cc:648] HybridClock initialized: now 1786250927504035 us; error 36 us; skew 500 ppm
I20260809 04:48:47.504432 2983 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:48:47 dist-test-slave-1jjb krb5kdc[2970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250927, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:47.507897 2983 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:48:47.509476 2983 webserver.cc:460] Webserver started at http://127.0.0.1:42955/ using document root <none> and password file <none>
I20260809 04:48:47.510111 2983 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:47.510257 2983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:47.510581 2983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:47.513119 2983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "d4edf6d30fe14cb98d24cb38ab35c2ac"
format_stamp: "Formatted at 2026-08-09 04:48:47 on dist-test-slave-1jjb"
server_key: "60dc6dc8237e9bce51c288917857a9a8"
server_key_iv: "f13a8ed825f5ec593dbb9c62d6abeaa0"
server_key_version: "encryptionkey@0"
I20260809 04:48:47.513811 2983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "d4edf6d30fe14cb98d24cb38ab35c2ac"
format_stamp: "Formatted at 2026-08-09 04:48:47 on dist-test-slave-1jjb"
server_key: "60dc6dc8237e9bce51c288917857a9a8"
server_key_iv: "f13a8ed825f5ec593dbb9c62d6abeaa0"
server_key_version: "encryptionkey@0"
I20260809 04:48:47.518179 2983 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:48:47.521335 2999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:47.522688 2983 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:48:47.522861 2983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "d4edf6d30fe14cb98d24cb38ab35c2ac"
format_stamp: "Formatted at 2026-08-09 04:48:47 on dist-test-slave-1jjb"
server_key: "60dc6dc8237e9bce51c288917857a9a8"
server_key_iv: "f13a8ed825f5ec593dbb9c62d6abeaa0"
server_key_version: "encryptionkey@0"
I20260809 04:48:47.523041 2983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:47.543881 2983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:47.547894 2983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:47.548218 2983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:47.565582 2983 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45177
I20260809 04:48:47.565621 3053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45177 every 8 connection(s)
I20260809 04:48:47.566700 2983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:48:47.567344 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 2983
I20260809 04:48:47.567559 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:48:47.568341 13224 external_mini_cluster.cc:1428] Setting key 4af647e20954b1e47be8a2bb527d8382
I20260809 04:48:47.570634 3054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:47.579982 3054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4edf6d30fe14cb98d24cb38ab35c2ac: Bootstrap starting.
Aug 09 04:48:47 dist-test-slave-1jjb krb5kdc[2970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 04:48:47.583235 3054 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4edf6d30fe14cb98d24cb38ab35c2ac: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:47.584337 3054 log.cc:824] T 00000000000000000000000000000000 P d4edf6d30fe14cb98d24cb38ab35c2ac: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:47.587517 3054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4edf6d30fe14cb98d24cb38ab35c2ac: No bootstrap required, opened a new log
I20260809 04:48:47.589092 3058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:47.570164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41767 (local address 127.0.0.1:45177)
0809 04:48:47.570932 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:47.570941 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:47.570974 (+ 33us) server_negotiation.cc:407] Connection header received
0809 04:48:47.572560 (+ 1586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:47.572580 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:47.572912 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:47.573253 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:47.574510 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:47.575606 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:47.576753 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:47.577351 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:47.577554 (+ 203us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:47.581347 (+ 3793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:47.581354 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:47.581365 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:47.581396 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:47.584396 (+ 3000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:47.585439 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:47.585445 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:47.585454 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:47.585702 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:47.586161 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:47.586166 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:47.586169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:47.586830 (+ 661us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:47.587169 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:47.587578 (+ 409us) server_negotiation.cc:299] Negotiation successful
0809 04:48:47.587893 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":345,"threads_started":1}
I20260809 04:48:47.593158 13224 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:48:47.597318 3054 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4edf6d30fe14cb98d24cb38ab35c2ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4edf6d30fe14cb98d24cb38ab35c2ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45177 } }
I20260809 04:48:47.597750 3054 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4edf6d30fe14cb98d24cb38ab35c2ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:47.597858 3054 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4edf6d30fe14cb98d24cb38ab35c2ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4edf6d30fe14cb98d24cb38ab35c2ac, State: Initialized, Role: FOLLOWER
I20260809 04:48:47.598521 3054 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4edf6d30fe14cb98d24cb38ab35c2ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4edf6d30fe14cb98d24cb38ab35c2ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45177 } }
I20260809 04:48:47.598809 3054 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4edf6d30fe14cb98d24cb38ab35c2ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:48:47.598942 3054 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4edf6d30fe14cb98d24cb38ab35c2ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:48:47.599108 3054 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4edf6d30fe14cb98d24cb38ab35c2ac [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:47.601258 3054 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4edf6d30fe14cb98d24cb38ab35c2ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4edf6d30fe14cb98d24cb38ab35c2ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45177 } }
I20260809 04:48:47.601696 3054 leader_election.cc:302] T 00000000000000000000000000000000 P d4edf6d30fe14cb98d24cb38ab35c2ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4edf6d30fe14cb98d24cb38ab35c2ac; no voters:
I20260809 04:48:47.602279 3054 leader_election.cc:288] T 00000000000000000000000000000000 P d4edf6d30fe14cb98d24cb38ab35c2ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:48:47.602358 3066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4edf6d30fe14cb98d24cb38ab35c2ac [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:48:47.604547 3054 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4edf6d30fe14cb98d24cb38ab35c2ac [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:48:47.607087 3066 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4edf6d30fe14cb98d24cb38ab35c2ac [term 1 LEADER]: Becoming Leader. State: Replica: d4edf6d30fe14cb98d24cb38ab35c2ac, State: Running, Role: LEADER
I20260809 04:48:47.608026 3066 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4edf6d30fe14cb98d24cb38ab35c2ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4edf6d30fe14cb98d24cb38ab35c2ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45177 } }
I20260809 04:48:47.613425 3067 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4edf6d30fe14cb98d24cb38ab35c2ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4edf6d30fe14cb98d24cb38ab35c2ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4edf6d30fe14cb98d24cb38ab35c2ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45177 } } }
I20260809 04:48:47.613780 3067 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4edf6d30fe14cb98d24cb38ab35c2ac [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:47.613399 3068 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4edf6d30fe14cb98d24cb38ab35c2ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4edf6d30fe14cb98d24cb38ab35c2ac. Latest consensus state: current_term: 1 leader_uuid: "d4edf6d30fe14cb98d24cb38ab35c2ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4edf6d30fe14cb98d24cb38ab35c2ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45177 } } }
I20260809 04:48:47.614323 3068 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4edf6d30fe14cb98d24cb38ab35c2ac [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:47.616547 3078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:48:47.616643 3058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:47.598671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47463 (local address 127.0.0.1:45177)
0809 04:48:47.598840 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:47.598846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:47.599151 (+ 305us) server_negotiation.cc:407] Connection header received
0809 04:48:47.599335 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:47.599341 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:47.599414 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:47.599521 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:47.600859 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:47.601562 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:47.603059 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:47.603555 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:47.603621 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:47.606323 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:47.606334 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:47.606346 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:47.606396 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:47.611171 (+ 4775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:47.612487 (+ 1316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:47.612495 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:47.612501 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:47.612775 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:47.613561 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:47.613570 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:47.613575 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:47.614054 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:47.614415 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:47.615807 (+ 1392us) server_negotiation.cc:299] Negotiation successful
0809 04:48:47.616034 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:48:47.623200 3078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:48:47.630904 3078 catalog_manager.cc:1347] Generated new cluster ID: f2e54580f6c147e4935d046eaa779c24
I20260809 04:48:47.631006 3078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:48:47.642086 3078 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:48:47.643090 3078 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:48:47.649879 3078 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4edf6d30fe14cb98d24cb38ab35c2ac: Generated new TSK 0
I20260809 04:48:47.650545 3078 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:48:47.655404 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 2983
I20260809 04:48:47.656837 3048 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:47.736536 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 2983
W20260809 04:48:47.757304 2967 connection.cc:535] client connection to 127.0.0.1:45177 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45177): Connection reset by peer (error 104)
2026-08-09T04:48:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1873 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:48:47 dist-test-slave-1jjb krb5kdc[3090](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:48:47 dist-test-slave-1jjb krb5kdc[3090](info): set up 2 sockets
Aug 09 04:48:47 dist-test-slave-1jjb krb5kdc[3090](info): commencing operation
krb5kdc: starting...
W20260809 04:48:48.789233 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.003s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:48:48 dist-test-slave-1jjb krb5kdc[3090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:48:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:48:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:48:49.034885 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40191
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:48:49.339524 3104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:49.339820 3104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:49.340001 3104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:49.346088 3104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:48:49.346203 3104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:49.346292 3104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:49.346375 3104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:48:49.346431 3104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:48:49.355240 3104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40191
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:49.356384 3104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:49.357569 3104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:49.368355 3109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:49.368316 3110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:49.368477 3112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:49.369043 3104 server_base.cc:1034] running on GCE node
I20260809 04:48:49.369796 3104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:49.371021 3104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:49.372241 3104 hybrid_clock.cc:648] HybridClock initialized: now 1786250929372202 us; error 41 us; skew 500 ppm
I20260809 04:48:49.372556 3104 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:48:49 dist-test-slave-1jjb krb5kdc[3090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250929, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:49.375524 3104 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:48:49.376950 3104 webserver.cc:460] Webserver started at http://127.0.0.1:39171/ using document root <none> and password file <none>
I20260809 04:48:49.377545 3104 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:49.377676 3104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:49.377940 3104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:49.380642 3104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "935c97641b374dbaab9ddda52862e6bf"
format_stamp: "Formatted at 2026-08-09 04:48:49 on dist-test-slave-1jjb"
server_key: "4ac9a9843e63ecd7b1b2aff6d67ad4e5"
server_key_iv: "f3609d28337e775d4b46f23f2ef39f2d"
server_key_version: "encryptionkey@0"
I20260809 04:48:49.381300 3104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "935c97641b374dbaab9ddda52862e6bf"
format_stamp: "Formatted at 2026-08-09 04:48:49 on dist-test-slave-1jjb"
server_key: "4ac9a9843e63ecd7b1b2aff6d67ad4e5"
server_key_iv: "f3609d28337e775d4b46f23f2ef39f2d"
server_key_version: "encryptionkey@0"
I20260809 04:48:49.385792 3104 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:48:49.388815 3120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:49.390031 3104 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:48:49.390211 3104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "935c97641b374dbaab9ddda52862e6bf"
format_stamp: "Formatted at 2026-08-09 04:48:49 on dist-test-slave-1jjb"
server_key: "4ac9a9843e63ecd7b1b2aff6d67ad4e5"
server_key_iv: "f3609d28337e775d4b46f23f2ef39f2d"
server_key_version: "encryptionkey@0"
I20260809 04:48:49.390404 3104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:49.400310 3104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:49.404450 3104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:49.404752 3104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:49.422871 3104 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40191
I20260809 04:48:49.422900 3174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40191 every 8 connection(s)
I20260809 04:48:49.424211 3104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:48:49.426182 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 3104
I20260809 04:48:49.426396 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:48:49.426821 13224 external_mini_cluster.cc:1428] Setting key 60e383ae1449c6fd9b9885dcfc50fecf
I20260809 04:48:49.427596 3175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:49.439803 3178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:49.428498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45223 (local address 127.0.0.1:40191)
0809 04:48:49.429263 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:49.429277 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:49.429342 (+ 65us) server_negotiation.cc:407] Connection header received
0809 04:48:49.430955 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:49.430978 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:49.431403 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:49.431767 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:49.432245 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:49.434078 (+ 1833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:49.435264 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:49.436167 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:49.436339 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:49.437416 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:49.437425 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:49.437440 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:49.438046 (+ 606us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:49.438135 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:49.438165 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:49.438313 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:49.438550 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 04:48:49.438697 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":309,"threads_started":1}
I20260809 04:48:49.440752 3175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 935c97641b374dbaab9ddda52862e6bf: Bootstrap starting.
I20260809 04:48:49.443502 13224 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:48:49.443924 3175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 935c97641b374dbaab9ddda52862e6bf: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:49.444969 3175 log.cc:824] T 00000000000000000000000000000000 P 935c97641b374dbaab9ddda52862e6bf: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:49.447793 3175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 935c97641b374dbaab9ddda52862e6bf: No bootstrap required, opened a new log
I20260809 04:48:49.454420 3178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:49.448046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40375 (local address 127.0.0.1:40191)
0809 04:48:49.448231 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:49.448236 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:49.448659 (+ 423us) server_negotiation.cc:407] Connection header received
0809 04:48:49.448783 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:49.448788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:49.448848 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:49.448954 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:49.449524 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:49.450690 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:49.451943 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:49.452632 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:49.452754 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:49.453725 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:49.453731 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:49.453735 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:49.453752 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:49.453793 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:49.453810 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:49.453919 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:49.454061 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 04:48:49.454102 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 04:48:49.456426 3175 raft_consensus.cc:348] T 00000000000000000000000000000000 P 935c97641b374dbaab9ddda52862e6bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "935c97641b374dbaab9ddda52862e6bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40191 } }
I20260809 04:48:49.456732 3175 raft_consensus.cc:374] T 00000000000000000000000000000000 P 935c97641b374dbaab9ddda52862e6bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:49.456806 3175 raft_consensus.cc:729] T 00000000000000000000000000000000 P 935c97641b374dbaab9ddda52862e6bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 935c97641b374dbaab9ddda52862e6bf, State: Initialized, Role: FOLLOWER
I20260809 04:48:49.457195 3175 consensus_queue.cc:260] T 00000000000000000000000000000000 P 935c97641b374dbaab9ddda52862e6bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "935c97641b374dbaab9ddda52862e6bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40191 } }
I20260809 04:48:49.457361 3175 raft_consensus.cc:388] T 00000000000000000000000000000000 P 935c97641b374dbaab9ddda52862e6bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:48:49.457455 3175 raft_consensus.cc:482] T 00000000000000000000000000000000 P 935c97641b374dbaab9ddda52862e6bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:48:49.457547 3175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 935c97641b374dbaab9ddda52862e6bf [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:49.459391 3175 raft_consensus.cc:504] T 00000000000000000000000000000000 P 935c97641b374dbaab9ddda52862e6bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "935c97641b374dbaab9ddda52862e6bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40191 } }
I20260809 04:48:49.459805 3175 leader_election.cc:302] T 00000000000000000000000000000000 P 935c97641b374dbaab9ddda52862e6bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 935c97641b374dbaab9ddda52862e6bf; no voters:
I20260809 04:48:49.460561 3175 leader_election.cc:288] T 00000000000000000000000000000000 P 935c97641b374dbaab9ddda52862e6bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:48:49.460660 3186 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 935c97641b374dbaab9ddda52862e6bf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:48:49.461480 3186 raft_consensus.cc:686] T 00000000000000000000000000000000 P 935c97641b374dbaab9ddda52862e6bf [term 1 LEADER]: Becoming Leader. State: Replica: 935c97641b374dbaab9ddda52862e6bf, State: Running, Role: LEADER
I20260809 04:48:49.461958 3186 consensus_queue.cc:237] T 00000000000000000000000000000000 P 935c97641b374dbaab9ddda52862e6bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "935c97641b374dbaab9ddda52862e6bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40191 } }
I20260809 04:48:49.462582 3175 sys_catalog.cc:564] T 00000000000000000000000000000000 P 935c97641b374dbaab9ddda52862e6bf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:48:49.464568 3187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 935c97641b374dbaab9ddda52862e6bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "935c97641b374dbaab9ddda52862e6bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "935c97641b374dbaab9ddda52862e6bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40191 } } }
I20260809 04:48:49.464995 3187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 935c97641b374dbaab9ddda52862e6bf [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:49.465215 3188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 935c97641b374dbaab9ddda52862e6bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 935c97641b374dbaab9ddda52862e6bf. Latest consensus state: current_term: 1 leader_uuid: "935c97641b374dbaab9ddda52862e6bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "935c97641b374dbaab9ddda52862e6bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40191 } } }
I20260809 04:48:49.465533 3188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 935c97641b374dbaab9ddda52862e6bf [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:49.467065 3195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:48:49.472937 3195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:48:49.480012 3204 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 04:48:49.482431 3195 catalog_manager.cc:1347] Generated new cluster ID: e2fbee9621e4478caa521f4c94745874
I20260809 04:48:49.482527 3195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:48:49.506438 3195 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:48:49.507987 3195 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:48:49.515407 3195 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 935c97641b374dbaab9ddda52862e6bf: Generated new TSK 0
I20260809 04:48:49.516278 3195 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:48:49.528092 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 3104
W20260809 04:48:49.529358 3170 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45223
W20260809 04:48:49.530097 13224 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 3104: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:48:49.530274 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 3104
W20260809 04:48:49.551045 3088 connection.cc:535] client connection to 127.0.0.1:40191 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40191): Connection reset by peer (error 104)
2026-08-09T04:48:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1792 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:48:49 dist-test-slave-1jjb krb5kdc[3208](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:48:49 dist-test-slave-1jjb krb5kdc[3208](info): set up 2 sockets
Aug 09 04:48:49 dist-test-slave-1jjb krb5kdc[3208](info): commencing operation
krb5kdc: starting...
W20260809 04:48:50.608769 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.007s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:48:50 dist-test-slave-1jjb krb5kdc[3208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:48:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:48:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:48:50.869679 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42489
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:48:51.184065 3222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:51.184332 3222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:51.184526 3222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:51.191018 3222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:48:51.191133 3222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:51.191207 3222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:51.191286 3222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:48:51.191351 3222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:48:51.200220 3222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42489
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:51.201308 3222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:51.202457 3222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:51.213534 3227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:51.213521 3228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:51.213542 3230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:51.214424 3222 server_base.cc:1034] running on GCE node
I20260809 04:48:51.215276 3222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:51.216790 3222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:51.218042 3222 hybrid_clock.cc:648] HybridClock initialized: now 1786250931217949 us; error 80 us; skew 500 ppm
I20260809 04:48:51.218436 3222 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:48:51 dist-test-slave-1jjb krb5kdc[3208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250931, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:51.226483 3222 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:48:51.227828 3222 webserver.cc:460] Webserver started at http://127.0.0.1:40795/ using document root <none> and password file <none>
I20260809 04:48:51.228528 3222 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:51.228682 3222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:51.228981 3222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:51.231992 3222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "4f56cb2b8e934f1d95799c9fa9a2b537"
format_stamp: "Formatted at 2026-08-09 04:48:51 on dist-test-slave-1jjb"
server_key: "df7ef0b3bd21a873fb090451de02b135"
server_key_iv: "4b1cbacf77c753bfd14c4e53f600aebc"
server_key_version: "encryptionkey@0"
I20260809 04:48:51.232827 3222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "4f56cb2b8e934f1d95799c9fa9a2b537"
format_stamp: "Formatted at 2026-08-09 04:48:51 on dist-test-slave-1jjb"
server_key: "df7ef0b3bd21a873fb090451de02b135"
server_key_iv: "4b1cbacf77c753bfd14c4e53f600aebc"
server_key_version: "encryptionkey@0"
I20260809 04:48:51.237615 3222 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 04:48:51.240937 3238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:51.242105 3222 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:48:51.242287 3222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "4f56cb2b8e934f1d95799c9fa9a2b537"
format_stamp: "Formatted at 2026-08-09 04:48:51 on dist-test-slave-1jjb"
server_key: "df7ef0b3bd21a873fb090451de02b135"
server_key_iv: "4b1cbacf77c753bfd14c4e53f600aebc"
server_key_version: "encryptionkey@0"
I20260809 04:48:51.242475 3222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:51.247576 3222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:51.251686 3222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:51.251958 3222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:51.268679 3222 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42489
I20260809 04:48:51.268718 3290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42489 every 8 connection(s)
I20260809 04:48:51.270005 3222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:48:51.271495 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 3222
I20260809 04:48:51.271656 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:48:51.272112 13224 external_mini_cluster.cc:1428] Setting key f554da99970b8259d1232e7bf4289b1f
I20260809 04:48:51.273906 3291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:51.280242 3294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:51.273808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37881 (local address 127.0.0.1:42489)
0809 04:48:51.274610 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:51.274623 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:51.274674 (+ 51us) server_negotiation.cc:407] Connection header received
0809 04:48:51.276647 (+ 1973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:51.276688 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:51.277099 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:51.277513 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:51.278375 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:51.278382 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:51.278396 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:51.278410 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:51.278504 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:51.278536 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:51.278732 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:51.278860 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 04:48:51.279055 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":354,"threads_started":1}
I20260809 04:48:51.283558 13224 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:48:51.286638 3291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f56cb2b8e934f1d95799c9fa9a2b537: Bootstrap starting.
I20260809 04:48:51.289963 3294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:51.287759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59053 (local address 127.0.0.1:42489)
0809 04:48:51.287933 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:51.287939 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:51.288263 (+ 324us) server_negotiation.cc:407] Connection header received
0809 04:48:51.288357 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:51.288363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:51.288412 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:51.288526 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:51.289326 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:51.289331 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:51.289336 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:51.289352 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:51.289394 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:51.289411 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:51.289531 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:51.289606 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 04:48:51.289659 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 04:48:51.290042 3291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f56cb2b8e934f1d95799c9fa9a2b537: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:51.291257 3291 log.cc:824] T 00000000000000000000000000000000 P 4f56cb2b8e934f1d95799c9fa9a2b537: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:51.293599 3291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f56cb2b8e934f1d95799c9fa9a2b537: No bootstrap required, opened a new log
I20260809 04:48:51.300494 3291 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f56cb2b8e934f1d95799c9fa9a2b537 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f56cb2b8e934f1d95799c9fa9a2b537" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42489 } }
I20260809 04:48:51.300808 3291 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f56cb2b8e934f1d95799c9fa9a2b537 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:51.300879 3291 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f56cb2b8e934f1d95799c9fa9a2b537 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f56cb2b8e934f1d95799c9fa9a2b537, State: Initialized, Role: FOLLOWER
I20260809 04:48:51.301393 3291 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f56cb2b8e934f1d95799c9fa9a2b537 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f56cb2b8e934f1d95799c9fa9a2b537" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42489 } }
I20260809 04:48:51.301605 3291 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f56cb2b8e934f1d95799c9fa9a2b537 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:48:51.301739 3291 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f56cb2b8e934f1d95799c9fa9a2b537 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:48:51.301927 3291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f56cb2b8e934f1d95799c9fa9a2b537 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:51.304193 3291 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f56cb2b8e934f1d95799c9fa9a2b537 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f56cb2b8e934f1d95799c9fa9a2b537" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42489 } }
I20260809 04:48:51.304692 3291 leader_election.cc:302] T 00000000000000000000000000000000 P 4f56cb2b8e934f1d95799c9fa9a2b537 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f56cb2b8e934f1d95799c9fa9a2b537; no voters:
I20260809 04:48:51.305307 3291 leader_election.cc:288] T 00000000000000000000000000000000 P 4f56cb2b8e934f1d95799c9fa9a2b537 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:48:51.305490 3301 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f56cb2b8e934f1d95799c9fa9a2b537 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:48:51.306238 3301 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f56cb2b8e934f1d95799c9fa9a2b537 [term 1 LEADER]: Becoming Leader. State: Replica: 4f56cb2b8e934f1d95799c9fa9a2b537, State: Running, Role: LEADER
I20260809 04:48:51.306891 3301 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f56cb2b8e934f1d95799c9fa9a2b537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f56cb2b8e934f1d95799c9fa9a2b537" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42489 } }
I20260809 04:48:51.307387 3291 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f56cb2b8e934f1d95799c9fa9a2b537 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:48:51.310369 3303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f56cb2b8e934f1d95799c9fa9a2b537 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f56cb2b8e934f1d95799c9fa9a2b537" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f56cb2b8e934f1d95799c9fa9a2b537" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42489 } } }
I20260809 04:48:51.311143 3303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f56cb2b8e934f1d95799c9fa9a2b537 [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:51.310734 3304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f56cb2b8e934f1d95799c9fa9a2b537 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f56cb2b8e934f1d95799c9fa9a2b537. Latest consensus state: current_term: 1 leader_uuid: "4f56cb2b8e934f1d95799c9fa9a2b537" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f56cb2b8e934f1d95799c9fa9a2b537" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42489 } } }
I20260809 04:48:51.312505 3304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f56cb2b8e934f1d95799c9fa9a2b537 [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:51.319916 3311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:48:51.324505 3311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:48:51.332728 3311 catalog_manager.cc:1347] Generated new cluster ID: de468311a5ab4c1d939209119207894e
I20260809 04:48:51.332845 3311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:48:51.341761 3311 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:48:51.342160 3311 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:48:51.350389 3311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f56cb2b8e934f1d95799c9fa9a2b537: Generated new TSK 0
I20260809 04:48:51.351048 3311 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:48:51.357339 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 3222
W20260809 04:48:51.358282 3286 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37881
W20260809 04:48:51.358868 13224 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 3222: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:48:51.359001 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 3222
2026-08-09T04:48:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1827 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:48:51 dist-test-slave-1jjb krb5kdc[3325](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:48:51 dist-test-slave-1jjb krb5kdc[3325](info): set up 2 sockets
Aug 09 04:48:51 dist-test-slave-1jjb krb5kdc[3325](info): commencing operation
krb5kdc: starting...
W20260809 04:48:52.424293 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.008s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:48:52 dist-test-slave-1jjb krb5kdc[3325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:48:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:48:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:48:52.687672 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36379
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36379
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:48:53.004276 3338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:53.004585 3338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:53.004741 3338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:53.011510 3338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:48:53.011643 3338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:53.011732 3338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:53.011842 3338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:48:53.011915 3338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:48:53.020969 3338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:53.022113 3338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:53.023362 3338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:53.034475 3346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:53.034832 3344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:53.035071 3343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:53.035866 3338 server_base.cc:1034] running on GCE node
I20260809 04:48:53.036640 3338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:53.037956 3338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:53.039155 3338 hybrid_clock.cc:648] HybridClock initialized: now 1786250933039105 us; error 47 us; skew 500 ppm
I20260809 04:48:53.039496 3338 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:48:53 dist-test-slave-1jjb krb5kdc[3325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250933, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:53.042498 3338 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:48:53.043892 3338 webserver.cc:460] Webserver started at http://127.0.0.1:42165/ using document root <none> and password file <none>
I20260809 04:48:53.044637 3338 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:53.044775 3338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:53.045090 3338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:53.047763 3338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "eef98ed3b81a412983c1fa66fcd0a526"
format_stamp: "Formatted at 2026-08-09 04:48:53 on dist-test-slave-1jjb"
server_key: "3ea34fa55c9d5dc759ae718e02efef5a"
server_key_iv: "539a6dfb3fc94dd0f99513741b079364"
server_key_version: "encryptionkey@0"
I20260809 04:48:53.048518 3338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "eef98ed3b81a412983c1fa66fcd0a526"
format_stamp: "Formatted at 2026-08-09 04:48:53 on dist-test-slave-1jjb"
server_key: "3ea34fa55c9d5dc759ae718e02efef5a"
server_key_iv: "539a6dfb3fc94dd0f99513741b079364"
server_key_version: "encryptionkey@0"
I20260809 04:48:53.053503 3338 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 04:48:53.056643 3353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:53.057811 3338 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:48:53.057973 3338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "eef98ed3b81a412983c1fa66fcd0a526"
format_stamp: "Formatted at 2026-08-09 04:48:53 on dist-test-slave-1jjb"
server_key: "3ea34fa55c9d5dc759ae718e02efef5a"
server_key_iv: "539a6dfb3fc94dd0f99513741b079364"
server_key_version: "encryptionkey@0"
I20260809 04:48:53.058156 3338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:53.089622 3338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:53.093266 3338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:53.093535 3338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:53.111155 3338 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36379
I20260809 04:48:53.111232 3410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36379 every 8 connection(s)
I20260809 04:48:53.112435 3338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:48:53.114049 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 3338
I20260809 04:48:53.114233 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:48:53.114741 13224 external_mini_cluster.cc:1428] Setting key 1489658f76b777ed73845ba428c5c570
I20260809 04:48:53.116460 3411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:53.126884 3411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eef98ed3b81a412983c1fa66fcd0a526: Bootstrap starting.
Aug 09 04:48:53 dist-test-slave-1jjb krb5kdc[3325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 04:48:53.129884 3411 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eef98ed3b81a412983c1fa66fcd0a526: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:53.131048 3411 log.cc:824] T 00000000000000000000000000000000 P eef98ed3b81a412983c1fa66fcd0a526: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:53.133699 3411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eef98ed3b81a412983c1fa66fcd0a526: No bootstrap required, opened a new log
I20260809 04:48:53.136194 3415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:53.116524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40367 (local address 127.0.0.1:36379)
0809 04:48:53.117248 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:53.117259 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:53.117296 (+ 37us) server_negotiation.cc:407] Connection header received
0809 04:48:53.118788 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:53.118807 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:53.119132 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:53.119484 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:53.121100 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:53.122337 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:53.123606 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:53.124329 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:53.124522 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:53.128606 (+ 4084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:53.128614 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:53.128625 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:53.128664 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:53.131978 (+ 3314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:53.132779 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:53.132786 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:53.132791 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:53.132971 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:53.133406 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:53.133410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:53.133412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:53.134000 (+ 588us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:53.134343 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:53.134777 (+ 434us) server_negotiation.cc:299] Negotiation successful
0809 04:48:53.135182 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":333,"threads_started":1}
I20260809 04:48:53.139856 13224 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:48:53.141342 3411 raft_consensus.cc:348] T 00000000000000000000000000000000 P eef98ed3b81a412983c1fa66fcd0a526 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef98ed3b81a412983c1fa66fcd0a526" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36379 } }
I20260809 04:48:53.141619 3411 raft_consensus.cc:374] T 00000000000000000000000000000000 P eef98ed3b81a412983c1fa66fcd0a526 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:53.141752 3411 raft_consensus.cc:729] T 00000000000000000000000000000000 P eef98ed3b81a412983c1fa66fcd0a526 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eef98ed3b81a412983c1fa66fcd0a526, State: Initialized, Role: FOLLOWER
I20260809 04:48:53.142223 3411 consensus_queue.cc:260] T 00000000000000000000000000000000 P eef98ed3b81a412983c1fa66fcd0a526 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef98ed3b81a412983c1fa66fcd0a526" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36379 } }
I20260809 04:48:53.142454 3411 raft_consensus.cc:388] T 00000000000000000000000000000000 P eef98ed3b81a412983c1fa66fcd0a526 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:48:53.142601 3411 raft_consensus.cc:482] T 00000000000000000000000000000000 P eef98ed3b81a412983c1fa66fcd0a526 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:48:53.142794 3411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eef98ed3b81a412983c1fa66fcd0a526 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:53.145197 3411 raft_consensus.cc:504] T 00000000000000000000000000000000 P eef98ed3b81a412983c1fa66fcd0a526 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef98ed3b81a412983c1fa66fcd0a526" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36379 } }
I20260809 04:48:53.145751 3411 leader_election.cc:302] T 00000000000000000000000000000000 P eef98ed3b81a412983c1fa66fcd0a526 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eef98ed3b81a412983c1fa66fcd0a526; no voters:
I20260809 04:48:53.146435 3411 leader_election.cc:288] T 00000000000000000000000000000000 P eef98ed3b81a412983c1fa66fcd0a526 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:48:53.146512 3422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eef98ed3b81a412983c1fa66fcd0a526 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:48:53.148715 3411 sys_catalog.cc:564] T 00000000000000000000000000000000 P eef98ed3b81a412983c1fa66fcd0a526 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:48:53.152352 3422 raft_consensus.cc:686] T 00000000000000000000000000000000 P eef98ed3b81a412983c1fa66fcd0a526 [term 1 LEADER]: Becoming Leader. State: Replica: eef98ed3b81a412983c1fa66fcd0a526, State: Running, Role: LEADER
I20260809 04:48:53.153326 3422 consensus_queue.cc:237] T 00000000000000000000000000000000 P eef98ed3b81a412983c1fa66fcd0a526 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef98ed3b81a412983c1fa66fcd0a526" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36379 } }
I20260809 04:48:53.156323 3423 sys_catalog.cc:455] T 00000000000000000000000000000000 P eef98ed3b81a412983c1fa66fcd0a526 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eef98ed3b81a412983c1fa66fcd0a526" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef98ed3b81a412983c1fa66fcd0a526" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36379 } } }
I20260809 04:48:53.156903 3423 sys_catalog.cc:458] T 00000000000000000000000000000000 P eef98ed3b81a412983c1fa66fcd0a526 [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:53.156996 3424 sys_catalog.cc:455] T 00000000000000000000000000000000 P eef98ed3b81a412983c1fa66fcd0a526 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eef98ed3b81a412983c1fa66fcd0a526. Latest consensus state: current_term: 1 leader_uuid: "eef98ed3b81a412983c1fa66fcd0a526" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef98ed3b81a412983c1fa66fcd0a526" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36379 } } }
I20260809 04:48:53.157249 3424 sys_catalog.cc:458] T 00000000000000000000000000000000 P eef98ed3b81a412983c1fa66fcd0a526 [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:53.158103 3415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:53.144131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42463 (local address 127.0.0.1:36379)
0809 04:48:53.144324 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:53.144331 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:53.144651 (+ 320us) server_negotiation.cc:407] Connection header received
0809 04:48:53.144817 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:53.144822 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:53.144881 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:53.144972 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:53.146378 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:53.147445 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:53.148732 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:53.149226 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:53.149368 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:53.151790 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:53.151800 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:53.151804 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:53.151828 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:53.154656 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:53.155418 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:53.155425 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:53.155430 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:53.155601 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:53.156126 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:53.156130 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:53.156133 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:53.156476 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:53.156945 (+ 469us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:53.157579 (+ 634us) server_negotiation.cc:299] Negotiation successful
0809 04:48:53.157709 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 04:48:53.161319 3429 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:48:53.166767 3429 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:48:53.174670 3429 catalog_manager.cc:1347] Generated new cluster ID: 3855d1e70d514cfab31c4e5daaa45e2b
I20260809 04:48:53.174940 3429 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:48:53.182390 3429 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:48:53.184130 3429 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:48:53.201968 3429 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eef98ed3b81a412983c1fa66fcd0a526: Generated new TSK 0
I20260809 04:48:53.202872 3429 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:48:53.215605 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 3338
I20260809 04:48:53.216775 3406 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:53.301082 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 3338
2026-08-09T04:48:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1941 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:48:53 dist-test-slave-1jjb krb5kdc[3446](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:48:53 dist-test-slave-1jjb krb5kdc[3446](info): set up 2 sockets
Aug 09 04:48:53 dist-test-slave-1jjb krb5kdc[3446](info): commencing operation
krb5kdc: starting...
W20260809 04:48:54.396296 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:48:54 dist-test-slave-1jjb krb5kdc[3446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:48:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:48:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:48:54.666843 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44267
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:48:54.976409 3461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:54.976660 3461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:54.976819 3461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:54.983455 3461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:48:54.983544 3461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:54.983671 3461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:54.983776 3461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:48:54.983855 3461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:48:54.992974 3461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:54.994055 3461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:54.995170 3461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:55.006140 3466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:55.006210 3469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:55.006173 3467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:55.007381 3461 server_base.cc:1034] running on GCE node
I20260809 04:48:55.008169 3461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:55.009423 3461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:55.010629 3461 hybrid_clock.cc:648] HybridClock initialized: now 1786250935010576 us; error 67 us; skew 500 ppm
I20260809 04:48:55.011004 3461 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:48:55 dist-test-slave-1jjb krb5kdc[3446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250935, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:55.013993 3461 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:48:55.015398 3461 webserver.cc:460] Webserver started at http://127.0.0.1:33619/ using document root <none> and password file <none>
I20260809 04:48:55.016038 3461 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:55.016237 3461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:55.016541 3461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:55.019248 3461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "8d24561e566046f7a23a84432495dbb9"
format_stamp: "Formatted at 2026-08-09 04:48:55 on dist-test-slave-1jjb"
server_key: "326c5508ed394c905c18514e827e4992"
server_key_iv: "6bcfcddd5b3de8fcc48140811836c28f"
server_key_version: "encryptionkey@0"
I20260809 04:48:55.020054 3461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "8d24561e566046f7a23a84432495dbb9"
format_stamp: "Formatted at 2026-08-09 04:48:55 on dist-test-slave-1jjb"
server_key: "326c5508ed394c905c18514e827e4992"
server_key_iv: "6bcfcddd5b3de8fcc48140811836c28f"
server_key_version: "encryptionkey@0"
I20260809 04:48:55.024921 3461 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 04:48:55.028029 3478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:55.029281 3461 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:48:55.029470 3461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "8d24561e566046f7a23a84432495dbb9"
format_stamp: "Formatted at 2026-08-09 04:48:55 on dist-test-slave-1jjb"
server_key: "326c5508ed394c905c18514e827e4992"
server_key_iv: "6bcfcddd5b3de8fcc48140811836c28f"
server_key_version: "encryptionkey@0"
I20260809 04:48:55.029660 3461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:55.052838 3461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:55.056850 3461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:55.057165 3461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:55.076671 3461 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44267
I20260809 04:48:55.076712 3537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44267 every 8 connection(s)
I20260809 04:48:55.077978 3461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:48:55.081013 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 3461
I20260809 04:48:55.081195 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:48:55.081773 13224 external_mini_cluster.cc:1428] Setting key 18467f22c71366ba76327b64a85463b8
I20260809 04:48:55.081874 3538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:55.095304 3538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d24561e566046f7a23a84432495dbb9: Bootstrap starting.
I20260809 04:48:55.095630 3542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:55.083555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46975 (local address 127.0.0.1:44267)
0809 04:48:55.084430 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:55.084441 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:55.084485 (+ 44us) server_negotiation.cc:407] Connection header received
0809 04:48:55.086571 (+ 2086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:55.086592 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:55.086977 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:55.087286 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:55.087875 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:55.089313 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:55.091158 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:55.091858 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:55.092004 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:55.093314 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:55.093323 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:55.093337 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:55.093874 (+ 537us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:55.093953 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:55.093980 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:55.094073 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:55.094339 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 04:48:55.094613 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":453,"threads_started":1}
I20260809 04:48:55.098284 3538 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d24561e566046f7a23a84432495dbb9: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:55.099227 13224 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:48:55.099371 3538 log.cc:824] T 00000000000000000000000000000000 P 8d24561e566046f7a23a84432495dbb9: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:55.102092 3538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d24561e566046f7a23a84432495dbb9: No bootstrap required, opened a new log
I20260809 04:48:55.110261 3542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:55.103888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45563 (local address 127.0.0.1:44267)
0809 04:48:55.104060 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:55.104066 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:55.104404 (+ 338us) server_negotiation.cc:407] Connection header received
0809 04:48:55.104569 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:55.104572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:55.104629 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:55.104705 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:55.105337 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:55.106045 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:55.107788 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:55.108382 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:55.108471 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:55.109560 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:55.109564 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:55.109567 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:55.109583 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:55.109616 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:55.109630 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:55.109700 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:55.109866 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 04:48:55.109930 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 04:48:55.111757 3538 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d24561e566046f7a23a84432495dbb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d24561e566046f7a23a84432495dbb9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44267 } }
I20260809 04:48:55.112128 3538 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d24561e566046f7a23a84432495dbb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:55.112241 3538 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d24561e566046f7a23a84432495dbb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d24561e566046f7a23a84432495dbb9, State: Initialized, Role: FOLLOWER
I20260809 04:48:55.112820 3538 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d24561e566046f7a23a84432495dbb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d24561e566046f7a23a84432495dbb9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44267 } }
I20260809 04:48:55.113055 3538 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d24561e566046f7a23a84432495dbb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:48:55.113206 3538 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d24561e566046f7a23a84432495dbb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:48:55.113445 3538 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d24561e566046f7a23a84432495dbb9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:55.115579 3538 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d24561e566046f7a23a84432495dbb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d24561e566046f7a23a84432495dbb9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44267 } }
I20260809 04:48:55.116163 3538 leader_election.cc:302] T 00000000000000000000000000000000 P 8d24561e566046f7a23a84432495dbb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d24561e566046f7a23a84432495dbb9; no voters:
I20260809 04:48:55.116807 3538 leader_election.cc:288] T 00000000000000000000000000000000 P 8d24561e566046f7a23a84432495dbb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:48:55.116911 3549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d24561e566046f7a23a84432495dbb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:48:55.117677 3549 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d24561e566046f7a23a84432495dbb9 [term 1 LEADER]: Becoming Leader. State: Replica: 8d24561e566046f7a23a84432495dbb9, State: Running, Role: LEADER
I20260809 04:48:55.118245 3549 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d24561e566046f7a23a84432495dbb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d24561e566046f7a23a84432495dbb9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44267 } }
I20260809 04:48:55.118901 3538 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d24561e566046f7a23a84432495dbb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:48:55.121328 3550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d24561e566046f7a23a84432495dbb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d24561e566046f7a23a84432495dbb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d24561e566046f7a23a84432495dbb9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44267 } } }
I20260809 04:48:55.121663 3551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d24561e566046f7a23a84432495dbb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d24561e566046f7a23a84432495dbb9. Latest consensus state: current_term: 1 leader_uuid: "8d24561e566046f7a23a84432495dbb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d24561e566046f7a23a84432495dbb9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44267 } } }
I20260809 04:48:55.122702 3550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d24561e566046f7a23a84432495dbb9 [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:55.122702 3551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d24561e566046f7a23a84432495dbb9 [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:55.124395 3558 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:48:55.128679 3558 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:48:55.136490 3558 catalog_manager.cc:1347] Generated new cluster ID: b9a6bb71f654439fba62af4f1a9d7a3b
I20260809 04:48:55.136597 3558 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:48:55.152422 3558 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:48:55.153544 3558 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:48:55.161752 3558 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d24561e566046f7a23a84432495dbb9: Generated new TSK 0
I20260809 04:48:55.162384 3558 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:48:55.172830 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 3461
W20260809 04:48:55.173761 3531 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46975
W20260809 04:48:55.174367 13224 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 3461: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:48:55.174501 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 3461
2026-08-09T04:48:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1883 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:48:55 dist-test-slave-1jjb krb5kdc[3570](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:48:55 dist-test-slave-1jjb krb5kdc[3570](info): set up 2 sockets
Aug 09 04:48:55 dist-test-slave-1jjb krb5kdc[3570](info): commencing operation
krb5kdc: starting...
W20260809 04:48:56.283807 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.001s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:48:56 dist-test-slave-1jjb krb5kdc[3570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250936, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:48:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:48:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:48:56.547591 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35165
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35165
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:48:56.863361 3583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:56.863646 3583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:56.863824 3583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:56.870493 3583 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:48:56.870617 3583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:56.870719 3583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:56.870824 3583 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:48:56.870913 3583 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:48:56.879455 3583 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35165
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:56.880666 3583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:56.881861 3583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:56.893168 3591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:56.893136 3589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:56.893201 3588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:56.893879 3583 server_base.cc:1034] running on GCE node
I20260809 04:48:56.894593 3583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:56.895880 3583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:56.897125 3583 hybrid_clock.cc:648] HybridClock initialized: now 1786250936897059 us; error 44 us; skew 500 ppm
I20260809 04:48:56.897473 3583 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:48:56 dist-test-slave-1jjb krb5kdc[3570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250936, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:56.905932 3583 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:48:56.907251 3583 webserver.cc:460] Webserver started at http://127.0.0.1:42419/ using document root <none> and password file <none>
I20260809 04:48:56.907861 3583 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:56.907999 3583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:56.908308 3583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:56.911087 3583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "829c2d6982934869b5166438b4e78fb8"
format_stamp: "Formatted at 2026-08-09 04:48:56 on dist-test-slave-1jjb"
server_key: "915e92ccb709519d70dcc1b8cd43ed77"
server_key_iv: "8bbb9d94ec12d5a634b0fc6dbd8d3c8a"
server_key_version: "encryptionkey@0"
I20260809 04:48:56.911854 3583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "829c2d6982934869b5166438b4e78fb8"
format_stamp: "Formatted at 2026-08-09 04:48:56 on dist-test-slave-1jjb"
server_key: "915e92ccb709519d70dcc1b8cd43ed77"
server_key_iv: "8bbb9d94ec12d5a634b0fc6dbd8d3c8a"
server_key_version: "encryptionkey@0"
I20260809 04:48:56.916500 3583 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 04:48:56.919451 3598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:56.920706 3583 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:48:56.920897 3583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "829c2d6982934869b5166438b4e78fb8"
format_stamp: "Formatted at 2026-08-09 04:48:56 on dist-test-slave-1jjb"
server_key: "915e92ccb709519d70dcc1b8cd43ed77"
server_key_iv: "8bbb9d94ec12d5a634b0fc6dbd8d3c8a"
server_key_version: "encryptionkey@0"
I20260809 04:48:56.921094 3583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:56.926447 3583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:56.930140 3583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:56.930433 3583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:56.947233 3583 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35165
I20260809 04:48:56.947299 3653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35165 every 8 connection(s)
I20260809 04:48:56.948437 3583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:48:56.952296 3654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:56.952549 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 3583
I20260809 04:48:56.952708 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:48:56.953086 13224 external_mini_cluster.cc:1428] Setting key bb74b8e69d237bb75af6eb92e769c75d
I20260809 04:48:56.959466 3657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:56.954607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48947 (local address 127.0.0.1:35165)
0809 04:48:56.955319 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:56.955328 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:56.955369 (+ 41us) server_negotiation.cc:407] Connection header received
0809 04:48:56.956707 (+ 1338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:56.956725 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:56.956992 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:56.957327 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:56.958202 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:56.958207 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:56.958218 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:56.958229 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:56.958284 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:56.958299 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:56.958520 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:56.958637 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 04:48:56.958785 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":317,"threads_started":1}
I20260809 04:48:56.963013 13224 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:48:56.964202 3654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 829c2d6982934869b5166438b4e78fb8: Bootstrap starting.
I20260809 04:48:56.967037 3654 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 829c2d6982934869b5166438b4e78fb8: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:56.968484 3654 log.cc:824] T 00000000000000000000000000000000 P 829c2d6982934869b5166438b4e78fb8: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:56.969471 3657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:56.966953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44785 (local address 127.0.0.1:35165)
0809 04:48:56.967160 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:56.967167 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:56.967565 (+ 398us) server_negotiation.cc:407] Connection header received
0809 04:48:56.967719 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:56.967722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:56.967763 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:56.967832 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:56.968746 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:56.968749 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:56.968752 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:48:56.968765 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:56.968797 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:48:56.968809 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:56.968896 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:56.969120 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 04:48:56.969182 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:48:56.971429 3654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 829c2d6982934869b5166438b4e78fb8: No bootstrap required, opened a new log
I20260809 04:48:56.977824 3654 raft_consensus.cc:348] T 00000000000000000000000000000000 P 829c2d6982934869b5166438b4e78fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829c2d6982934869b5166438b4e78fb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35165 } }
I20260809 04:48:56.978159 3654 raft_consensus.cc:374] T 00000000000000000000000000000000 P 829c2d6982934869b5166438b4e78fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:56.978255 3654 raft_consensus.cc:729] T 00000000000000000000000000000000 P 829c2d6982934869b5166438b4e78fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 829c2d6982934869b5166438b4e78fb8, State: Initialized, Role: FOLLOWER
I20260809 04:48:56.978773 3654 consensus_queue.cc:260] T 00000000000000000000000000000000 P 829c2d6982934869b5166438b4e78fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829c2d6982934869b5166438b4e78fb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35165 } }
I20260809 04:48:56.978991 3654 raft_consensus.cc:388] T 00000000000000000000000000000000 P 829c2d6982934869b5166438b4e78fb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:48:56.979136 3654 raft_consensus.cc:482] T 00000000000000000000000000000000 P 829c2d6982934869b5166438b4e78fb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:48:56.979285 3654 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 829c2d6982934869b5166438b4e78fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:56.981606 3654 raft_consensus.cc:504] T 00000000000000000000000000000000 P 829c2d6982934869b5166438b4e78fb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829c2d6982934869b5166438b4e78fb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35165 } }
I20260809 04:48:56.982072 3654 leader_election.cc:302] T 00000000000000000000000000000000 P 829c2d6982934869b5166438b4e78fb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 829c2d6982934869b5166438b4e78fb8; no voters:
I20260809 04:48:56.982578 3654 leader_election.cc:288] T 00000000000000000000000000000000 P 829c2d6982934869b5166438b4e78fb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:48:56.982692 3665 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 829c2d6982934869b5166438b4e78fb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:48:56.983363 3665 raft_consensus.cc:686] T 00000000000000000000000000000000 P 829c2d6982934869b5166438b4e78fb8 [term 1 LEADER]: Becoming Leader. State: Replica: 829c2d6982934869b5166438b4e78fb8, State: Running, Role: LEADER
I20260809 04:48:56.983881 3665 consensus_queue.cc:237] T 00000000000000000000000000000000 P 829c2d6982934869b5166438b4e78fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829c2d6982934869b5166438b4e78fb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35165 } }
I20260809 04:48:56.984615 3654 sys_catalog.cc:564] T 00000000000000000000000000000000 P 829c2d6982934869b5166438b4e78fb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:48:56.987177 3666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 829c2d6982934869b5166438b4e78fb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "829c2d6982934869b5166438b4e78fb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829c2d6982934869b5166438b4e78fb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35165 } } }
I20260809 04:48:56.987128 3667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 829c2d6982934869b5166438b4e78fb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 829c2d6982934869b5166438b4e78fb8. Latest consensus state: current_term: 1 leader_uuid: "829c2d6982934869b5166438b4e78fb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829c2d6982934869b5166438b4e78fb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35165 } } }
I20260809 04:48:56.987577 3666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 829c2d6982934869b5166438b4e78fb8 [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:56.987617 3667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 829c2d6982934869b5166438b4e78fb8 [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:56.989935 3672 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:48:56.994220 3672 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:48:57.002055 3672 catalog_manager.cc:1347] Generated new cluster ID: 080598586c484b86b18e6c18ddd20ca8
I20260809 04:48:57.002146 3672 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:48:57.034559 3672 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:48:57.034929 3672 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:48:57.042480 3672 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 829c2d6982934869b5166438b4e78fb8: Generated new TSK 0
I20260809 04:48:57.043165 3672 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:48:57.050781 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 3583
W20260809 04:48:57.052053 3649 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48947
W20260809 04:48:57.052665 13224 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 3583: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:48:57.052793 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 3583
2026-08-09T04:48:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1870 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:48:57 dist-test-slave-1jjb krb5kdc[3688](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:48:57 dist-test-slave-1jjb krb5kdc[3688](info): set up 2 sockets
Aug 09 04:48:57 dist-test-slave-1jjb krb5kdc[3688](info): commencing operation
krb5kdc: starting...
W20260809 04:48:58.143131 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.001s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:48:58 dist-test-slave-1jjb krb5kdc[3688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:48:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:48:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:48:58.403507 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34269
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:48:58.719010 3701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:48:58.719256 3701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:48:58.719435 3701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:48:58.726294 3701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:48:58.726410 3701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:48:58.726532 3701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:48:58.726608 3701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:48:58.726678 3701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:48:58.736538 3701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:48:58.737736 3701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:48:58.738947 3701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:48:58.749684 3710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:58.749823 3708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:48:58.749722 3707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:48:58.750660 3701 server_base.cc:1034] running on GCE node
I20260809 04:48:58.751400 3701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:48:58.752666 3701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:48:58.753890 3701 hybrid_clock.cc:648] HybridClock initialized: now 1786250938753862 us; error 56 us; skew 500 ppm
I20260809 04:48:58.754202 3701 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:48:58 dist-test-slave-1jjb krb5kdc[3688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250938, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:48:58.757215 3701 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:48:58.758565 3701 webserver.cc:460] Webserver started at http://127.0.0.1:38309/ using document root <none> and password file <none>
I20260809 04:48:58.759202 3701 fs_manager.cc:359] Metadata directory not provided
I20260809 04:48:58.759351 3701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:48:58.759650 3701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:48:58.762573 3701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "a6935d036fd2496eb5cd686cdf43d779"
format_stamp: "Formatted at 2026-08-09 04:48:58 on dist-test-slave-1jjb"
server_key: "0401926b6a124a28c5badae131fb487b"
server_key_iv: "76df1cdf9e75952882494a36eb248e24"
server_key_version: "encryptionkey@0"
I20260809 04:48:58.763319 3701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "a6935d036fd2496eb5cd686cdf43d779"
format_stamp: "Formatted at 2026-08-09 04:48:58 on dist-test-slave-1jjb"
server_key: "0401926b6a124a28c5badae131fb487b"
server_key_iv: "76df1cdf9e75952882494a36eb248e24"
server_key_version: "encryptionkey@0"
I20260809 04:48:58.768317 3701 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 04:48:58.771566 3717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:48:58.772807 3701 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:48:58.772979 3701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "a6935d036fd2496eb5cd686cdf43d779"
format_stamp: "Formatted at 2026-08-09 04:48:58 on dist-test-slave-1jjb"
server_key: "0401926b6a124a28c5badae131fb487b"
server_key_iv: "76df1cdf9e75952882494a36eb248e24"
server_key_version: "encryptionkey@0"
I20260809 04:48:58.773155 3701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:48:58.801157 3701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:48:58.805431 3701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:48:58.805923 3701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:48:58.826159 3701 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34269
I20260809 04:48:58.826170 3771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34269 every 8 connection(s)
I20260809 04:48:58.827535 3701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:48:58.831264 3772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:48:58.836551 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 3701
I20260809 04:48:58.836777 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:48:58.837203 13224 external_mini_cluster.cc:1428] Setting key 2e2bb84140386002ef90f0cb1bd16251
I20260809 04:48:58.841681 3772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6935d036fd2496eb5cd686cdf43d779: Bootstrap starting.
I20260809 04:48:58.845068 3772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6935d036fd2496eb5cd686cdf43d779: Neither blocks nor log segments found. Creating new log.
I20260809 04:48:58.846200 3772 log.cc:824] T 00000000000000000000000000000000 P a6935d036fd2496eb5cd686cdf43d779: Log is configured to *not* fsync() on all Append() calls
I20260809 04:48:58.848795 3772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6935d036fd2496eb5cd686cdf43d779: No bootstrap required, opened a new log
Aug 09 04:48:58 dist-test-slave-1jjb krb5kdc[3688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 04:48:58.856768 3772 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6935d036fd2496eb5cd686cdf43d779 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6935d036fd2496eb5cd686cdf43d779" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34269 } }
I20260809 04:48:58.857159 3772 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6935d036fd2496eb5cd686cdf43d779 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:48:58.857261 3772 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6935d036fd2496eb5cd686cdf43d779 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6935d036fd2496eb5cd686cdf43d779, State: Initialized, Role: FOLLOWER
I20260809 04:48:58.857929 3772 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6935d036fd2496eb5cd686cdf43d779 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6935d036fd2496eb5cd686cdf43d779" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34269 } }
I20260809 04:48:58.858201 3772 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6935d036fd2496eb5cd686cdf43d779 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:48:58.858412 3772 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6935d036fd2496eb5cd686cdf43d779 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:48:58.858630 3772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6935d036fd2496eb5cd686cdf43d779 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:48:58.859540 3775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:58.839059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.174:44739 (local address 127.0.0.1:34269)
0809 04:48:58.839888 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:58.839898 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:58.839936 (+ 38us) server_negotiation.cc:407] Connection header received
0809 04:48:58.841726 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:58.841779 (+ 53us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:58.842153 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:58.842452 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:58.844208 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:58.845556 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:58.847128 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:58.847700 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:58.847891 (+ 191us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:58.852216 (+ 4325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:58.852223 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:58.852236 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:58.852272 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:58.855292 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:58.856115 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:58.856124 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:58.856132 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:58.856298 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:58.856739 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:58.856748 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:58.856750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:58.857388 (+ 638us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:58.857723 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:58.858194 (+ 471us) server_negotiation.cc:299] Negotiation successful
0809 04:48:58.858474 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":401,"threads_started":1}
I20260809 04:48:58.861928 3772 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6935d036fd2496eb5cd686cdf43d779 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6935d036fd2496eb5cd686cdf43d779" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34269 } }
I20260809 04:48:58.862529 3772 leader_election.cc:302] T 00000000000000000000000000000000 P a6935d036fd2496eb5cd686cdf43d779 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6935d036fd2496eb5cd686cdf43d779; no voters:
I20260809 04:48:58.863226 3772 leader_election.cc:288] T 00000000000000000000000000000000 P a6935d036fd2496eb5cd686cdf43d779 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:48:58.863255 3777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6935d036fd2496eb5cd686cdf43d779 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:48:58.863696 13224 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:48:58.864765 3777 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6935d036fd2496eb5cd686cdf43d779 [term 1 LEADER]: Becoming Leader. State: Replica: a6935d036fd2496eb5cd686cdf43d779, State: Running, Role: LEADER
I20260809 04:48:58.865204 3777 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6935d036fd2496eb5cd686cdf43d779 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6935d036fd2496eb5cd686cdf43d779" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34269 } }
I20260809 04:48:58.865332 3772 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6935d036fd2496eb5cd686cdf43d779 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:48:58.867681 3779 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6935d036fd2496eb5cd686cdf43d779 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6935d036fd2496eb5cd686cdf43d779. Latest consensus state: current_term: 1 leader_uuid: "a6935d036fd2496eb5cd686cdf43d779" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6935d036fd2496eb5cd686cdf43d779" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34269 } } }
I20260809 04:48:58.867689 3778 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6935d036fd2496eb5cd686cdf43d779 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6935d036fd2496eb5cd686cdf43d779" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6935d036fd2496eb5cd686cdf43d779" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34269 } } }
I20260809 04:48:58.868211 3778 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6935d036fd2496eb5cd686cdf43d779 [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:58.868211 3779 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6935d036fd2496eb5cd686cdf43d779 [sys.catalog]: This master's current role is: LEADER
I20260809 04:48:58.880975 3786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:48:58.886721 3786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:48:58.900693 3786 catalog_manager.cc:1347] Generated new cluster ID: a23ec95aedbe4646b0d78e750d11cf54
I20260809 04:48:58.900792 3786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:48:58.908806 3775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:48:58.882136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.174:44521 (local address 127.0.0.1:34269)
0809 04:48:58.882347 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 04:48:58.882355 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:48:58.892779 (+ 10424us) server_negotiation.cc:407] Connection header received
0809 04:48:58.893026 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:48:58.893031 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:48:58.893114 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:48:58.893204 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:48:58.895151 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:58.896676 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:58.898433 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:48:58.899112 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:48:58.899227 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:48:58.902465 (+ 3238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:48:58.902472 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:48:58.902477 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:48:58.902509 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:48:58.905781 (+ 3272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:58.906552 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:58.906556 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:58.906559 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:58.906699 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:48:58.907161 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:48:58.907166 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:48:58.907169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:48:58.907495 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:48:58.907712 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 04:48:58.908304 (+ 592us) server_negotiation.cc:299] Negotiation successful
0809 04:48:58.908475 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 04:48:58.912024 3801 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 04:48:58.915314 3786 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:48:58.916460 3786 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:48:58.927831 3786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6935d036fd2496eb5cd686cdf43d779: Generated new TSK 0
I20260809 04:48:58.928843 3786 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:48:58.935153 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 3701
I20260809 04:48:58.936316 3767 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:48:59.028029 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 3701
W20260809 04:48:59.048650 3685 connection.cc:535] client connection to 127.0.0.1:34269 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34269): Connection reset by peer (error 104)
2026-08-09T04:48:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1973 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:48:59 dist-test-slave-1jjb krb5kdc[3805](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:48:59 dist-test-slave-1jjb krb5kdc[3805](info): set up 2 sockets
Aug 09 04:48:59 dist-test-slave-1jjb krb5kdc[3805](info): commencing operation
krb5kdc: starting...
W20260809 04:49:00.136600 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.004s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:49:00 dist-test-slave-1jjb krb5kdc[3805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:49:00.407601 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42949
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:00.724798 3819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:00.725036 3819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:00.725116 3819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:00.731514 3819 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:00.731611 3819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:00.731663 3819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:00.731721 3819 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:00.731765 3819 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:00.740700 3819 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:49:00.741758 3819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:00.742969 3819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:00.754328 3825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:00.754516 3826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:00.754549 3828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:00.755518 3819 server_base.cc:1034] running on GCE node
I20260809 04:49:00.756251 3819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:00.757498 3819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:00.758720 3819 hybrid_clock.cc:648] HybridClock initialized: now 1786250940758693 us; error 50 us; skew 500 ppm
I20260809 04:49:00.759054 3819 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:49:00 dist-test-slave-1jjb krb5kdc[3805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250940, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:00.762071 3819 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:49:00.763420 3819 webserver.cc:460] Webserver started at http://127.0.0.1:41867/ using document root <none> and password file <none>
I20260809 04:49:00.764026 3819 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:00.764200 3819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:00.764501 3819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:00.767227 3819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "6c72f90270034d418ddf673c999e33ec"
format_stamp: "Formatted at 2026-08-09 04:49:00 on dist-test-slave-1jjb"
server_key: "17818042bde0cddf79e4c0973d3dd262"
server_key_iv: "5b43a63d205f4bb238612972d9f663c2"
server_key_version: "encryptionkey@0"
I20260809 04:49:00.767962 3819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "6c72f90270034d418ddf673c999e33ec"
format_stamp: "Formatted at 2026-08-09 04:49:00 on dist-test-slave-1jjb"
server_key: "17818042bde0cddf79e4c0973d3dd262"
server_key_iv: "5b43a63d205f4bb238612972d9f663c2"
server_key_version: "encryptionkey@0"
I20260809 04:49:00.772794 3819 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 04:49:00.776643 3837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:00.777832 3819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 04:49:00.778005 3819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "6c72f90270034d418ddf673c999e33ec"
format_stamp: "Formatted at 2026-08-09 04:49:00 on dist-test-slave-1jjb"
server_key: "17818042bde0cddf79e4c0973d3dd262"
server_key_iv: "5b43a63d205f4bb238612972d9f663c2"
server_key_version: "encryptionkey@0"
I20260809 04:49:00.778178 3819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:00.798583 3819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:00.802347 3819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:00.802621 3819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:00.820989 3819 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42949
I20260809 04:49:00.820992 3891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42949 every 8 connection(s)
I20260809 04:49:00.822366 3819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:00.825989 3892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:00.831379 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 3819
I20260809 04:49:00.831583 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:49:00.832002 13224 external_mini_cluster.cc:1428] Setting key 3dabaa6897cae7f553ceeabd1717f848
I20260809 04:49:00.836964 3892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c72f90270034d418ddf673c999e33ec: Bootstrap starting.
I20260809 04:49:00.840197 3892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c72f90270034d418ddf673c999e33ec: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:00.841329 3892 log.cc:824] T 00000000000000000000000000000000 P 6c72f90270034d418ddf673c999e33ec: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:00.844485 3892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c72f90270034d418ddf673c999e33ec: No bootstrap required, opened a new log
I20260809 04:49:00.845089 3895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:00.833786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.174:33121 (local address 127.0.0.1:42949)
0809 04:49:00.834742 (+ 956us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:00.834751 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:00.834790 (+ 39us) server_negotiation.cc:407] Connection header received
0809 04:49:00.836360 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:00.836386 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:00.836833 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:00.837168 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:00.837698 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:00.838911 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:00.840675 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:00.841309 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:00.841503 (+ 194us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:00.842821 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:00.842828 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:00.842839 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:00.843388 (+ 549us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:00.843455 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:00.843488 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:00.843603 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:00.843921 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 04:49:00.844115 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":763,"thread_start_us":456,"threads_started":1}
I20260809 04:49:00.848726 13224 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:00.852268 3892 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c72f90270034d418ddf673c999e33ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c72f90270034d418ddf673c999e33ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42949 } }
I20260809 04:49:00.852628 3892 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c72f90270034d418ddf673c999e33ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:00.852749 3892 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c72f90270034d418ddf673c999e33ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c72f90270034d418ddf673c999e33ec, State: Initialized, Role: FOLLOWER
I20260809 04:49:00.853431 3892 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c72f90270034d418ddf673c999e33ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c72f90270034d418ddf673c999e33ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42949 } }
I20260809 04:49:00.853719 3892 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c72f90270034d418ddf673c999e33ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:00.853951 3892 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c72f90270034d418ddf673c999e33ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:00.854144 3892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c72f90270034d418ddf673c999e33ec [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:00.856792 3892 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c72f90270034d418ddf673c999e33ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c72f90270034d418ddf673c999e33ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42949 } }
I20260809 04:49:00.857285 3892 leader_election.cc:302] T 00000000000000000000000000000000 P 6c72f90270034d418ddf673c999e33ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c72f90270034d418ddf673c999e33ec; no voters:
I20260809 04:49:00.857939 3892 leader_election.cc:288] T 00000000000000000000000000000000 P 6c72f90270034d418ddf673c999e33ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:00.858002 3902 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c72f90270034d418ddf673c999e33ec [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:00.859895 3895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:00.852907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.174:39811 (local address 127.0.0.1:42949)
0809 04:49:00.853075 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:00.853081 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:00.853674 (+ 593us) server_negotiation.cc:407] Connection header received
0809 04:49:00.853894 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:00.853897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:00.853962 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:00.854029 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:00.854641 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:00.855340 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:00.857049 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:00.857542 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:00.857629 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:00.858936 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:00.858940 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:00.858944 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:00.858961 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:00.859021 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:00.859038 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:00.859128 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:00.859535 (+ 407us) server_negotiation.cc:299] Negotiation successful
0809 04:49:00.859591 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 04:49:00.860250 3892 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c72f90270034d418ddf673c999e33ec [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:00.864161 3902 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c72f90270034d418ddf673c999e33ec [term 1 LEADER]: Becoming Leader. State: Replica: 6c72f90270034d418ddf673c999e33ec, State: Running, Role: LEADER
I20260809 04:49:00.864778 3902 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c72f90270034d418ddf673c999e33ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c72f90270034d418ddf673c999e33ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42949 } }
I20260809 04:49:00.867949 3904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c72f90270034d418ddf673c999e33ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c72f90270034d418ddf673c999e33ec. Latest consensus state: current_term: 1 leader_uuid: "6c72f90270034d418ddf673c999e33ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c72f90270034d418ddf673c999e33ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42949 } } }
I20260809 04:49:00.867969 3903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c72f90270034d418ddf673c999e33ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c72f90270034d418ddf673c999e33ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c72f90270034d418ddf673c999e33ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42949 } } }
I20260809 04:49:00.868391 3904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c72f90270034d418ddf673c999e33ec [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:00.868438 3903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c72f90270034d418ddf673c999e33ec [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:00.870452 3914 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:00.874732 3914 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:00.883181 3914 catalog_manager.cc:1347] Generated new cluster ID: ad4286e3bdfd4f6e9a267593f6487250
I20260809 04:49:00.883291 3914 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:00.892277 3914 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:00.893321 3914 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:00.899655 3914 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c72f90270034d418ddf673c999e33ec: Generated new TSK 0
I20260809 04:49:00.900372 3914 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:00.912688 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 3819
W20260809 04:49:00.913667 3887 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.174:33121
W20260809 04:49:00.914361 13224 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 3819: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:49:00.914494 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 3819
W20260809 04:49:00.935318 3803 connection.cc:535] client connection to 127.0.0.1:42949 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42949): Connection reset by peer (error 104)
2026-08-09T04:49:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1886 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:01 dist-test-slave-1jjb krb5kdc[3925](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:01 dist-test-slave-1jjb krb5kdc[3925](info): set up 2 sockets
Aug 09 04:49:01 dist-test-slave-1jjb krb5kdc[3925](info): commencing operation
krb5kdc: starting...
W20260809 04:49:02.015754 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.004s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:49:02 dist-test-slave-1jjb krb5kdc[3925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:49:02.280999 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39091
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:02.596365 3941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:02.596591 3941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:02.596668 3941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:02.603055 3941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:02.603137 3941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:02.603188 3941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:02.603236 3941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:02.603296 3941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:02.612092 3941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:49:02.613093 3941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:02.614169 3941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:02.624835 3947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:02.624893 3946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:02.624913 3949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:02.629439 3941 server_base.cc:1034] running on GCE node
I20260809 04:49:02.630095 3941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:02.631296 3941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:02.632507 3941 hybrid_clock.cc:648] HybridClock initialized: now 1786250942632456 us; error 29 us; skew 500 ppm
I20260809 04:49:02.632859 3941 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:49:02 dist-test-slave-1jjb krb5kdc[3925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250942, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:02.639050 3941 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:49:02.640651 3941 webserver.cc:460] Webserver started at http://127.0.0.1:44303/ using document root <none> and password file <none>
I20260809 04:49:02.641315 3941 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:02.641455 3941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:02.641779 3941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:02.644487 3941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "bd37e2bdeeb84f27810d27d79952db82"
format_stamp: "Formatted at 2026-08-09 04:49:02 on dist-test-slave-1jjb"
server_key: "916f9a45b621aa5221fcc410c9d6758c"
server_key_iv: "b04053643539d36359ddda0f1d30e863"
server_key_version: "encryptionkey@0"
I20260809 04:49:02.645187 3941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "bd37e2bdeeb84f27810d27d79952db82"
format_stamp: "Formatted at 2026-08-09 04:49:02 on dist-test-slave-1jjb"
server_key: "916f9a45b621aa5221fcc410c9d6758c"
server_key_iv: "b04053643539d36359ddda0f1d30e863"
server_key_version: "encryptionkey@0"
I20260809 04:49:02.649667 3941 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:49:02.652876 3956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:02.654019 3941 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 04:49:02.654204 3941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "bd37e2bdeeb84f27810d27d79952db82"
format_stamp: "Formatted at 2026-08-09 04:49:02 on dist-test-slave-1jjb"
server_key: "916f9a45b621aa5221fcc410c9d6758c"
server_key_iv: "b04053643539d36359ddda0f1d30e863"
server_key_version: "encryptionkey@0"
I20260809 04:49:02.654438 3941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:02.659415 3941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:02.662891 3941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:02.663153 3941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:02.680532 3941 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39091
I20260809 04:49:02.680611 4009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39091 every 8 connection(s)
I20260809 04:49:02.681748 3941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:02.683866 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 3941
I20260809 04:49:02.684038 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:49:02.684450 13224 external_mini_cluster.cc:1428] Setting key bb45b06f9c0b80780bd6ee3ae3fc5fa6
I20260809 04:49:02.685281 4010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:02.692232 4013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:02.686123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.174:34931 (local address 127.0.0.1:39091)
0809 04:49:02.686890 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:02.686904 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:02.686955 (+ 51us) server_negotiation.cc:407] Connection header received
0809 04:49:02.688716 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:02.688739 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:02.689157 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:02.689533 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:02.690386 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:02.690394 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:02.690408 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:02.690424 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:02.690555 (+ 131us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:02.690584 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:02.690809 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:02.690942 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 04:49:02.691135 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":305,"threads_started":1}
I20260809 04:49:02.695289 13224 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:02.699127 4010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd37e2bdeeb84f27810d27d79952db82: Bootstrap starting.
I20260809 04:49:02.702455 4013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:02.700239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.174:41791 (local address 127.0.0.1:39091)
0809 04:49:02.700378 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:02.700385 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:02.700789 (+ 404us) server_negotiation.cc:407] Connection header received
0809 04:49:02.700914 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:02.700920 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:02.700981 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:02.701095 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:02.701894 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:02.701900 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:02.701907 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:02.701925 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:02.701969 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:02.701993 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:02.702094 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:02.702173 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 04:49:02.702221 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:49:02.702834 4010 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd37e2bdeeb84f27810d27d79952db82: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:02.704030 4010 log.cc:824] T 00000000000000000000000000000000 P bd37e2bdeeb84f27810d27d79952db82: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:02.706259 4010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd37e2bdeeb84f27810d27d79952db82: No bootstrap required, opened a new log
I20260809 04:49:02.713020 4010 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd37e2bdeeb84f27810d27d79952db82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd37e2bdeeb84f27810d27d79952db82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39091 } }
I20260809 04:49:02.713394 4010 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd37e2bdeeb84f27810d27d79952db82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:02.713526 4010 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd37e2bdeeb84f27810d27d79952db82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd37e2bdeeb84f27810d27d79952db82, State: Initialized, Role: FOLLOWER
I20260809 04:49:02.714166 4010 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd37e2bdeeb84f27810d27d79952db82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd37e2bdeeb84f27810d27d79952db82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39091 } }
I20260809 04:49:02.714358 4010 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd37e2bdeeb84f27810d27d79952db82 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:02.714458 4010 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd37e2bdeeb84f27810d27d79952db82 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:02.714570 4010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd37e2bdeeb84f27810d27d79952db82 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:02.716533 4010 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd37e2bdeeb84f27810d27d79952db82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd37e2bdeeb84f27810d27d79952db82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39091 } }
I20260809 04:49:02.716924 4010 leader_election.cc:302] T 00000000000000000000000000000000 P bd37e2bdeeb84f27810d27d79952db82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd37e2bdeeb84f27810d27d79952db82; no voters:
I20260809 04:49:02.717351 4010 leader_election.cc:288] T 00000000000000000000000000000000 P bd37e2bdeeb84f27810d27d79952db82 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:02.717514 4021 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd37e2bdeeb84f27810d27d79952db82 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:02.718151 4021 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd37e2bdeeb84f27810d27d79952db82 [term 1 LEADER]: Becoming Leader. State: Replica: bd37e2bdeeb84f27810d27d79952db82, State: Running, Role: LEADER
I20260809 04:49:02.718679 4021 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd37e2bdeeb84f27810d27d79952db82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd37e2bdeeb84f27810d27d79952db82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39091 } }
I20260809 04:49:02.719254 4010 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd37e2bdeeb84f27810d27d79952db82 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:02.721009 4023 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd37e2bdeeb84f27810d27d79952db82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd37e2bdeeb84f27810d27d79952db82. Latest consensus state: current_term: 1 leader_uuid: "bd37e2bdeeb84f27810d27d79952db82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd37e2bdeeb84f27810d27d79952db82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39091 } } }
I20260809 04:49:02.721504 4023 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd37e2bdeeb84f27810d27d79952db82 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:02.721426 4022 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd37e2bdeeb84f27810d27d79952db82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd37e2bdeeb84f27810d27d79952db82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd37e2bdeeb84f27810d27d79952db82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39091 } } }
I20260809 04:49:02.721741 4022 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd37e2bdeeb84f27810d27d79952db82 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:02.723590 4026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:02.728621 4026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:02.736289 4026 catalog_manager.cc:1347] Generated new cluster ID: cecc02e533dd438c9a7c0f58c0aa40e1
I20260809 04:49:02.736382 4026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:02.761669 4026 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:02.762033 4026 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:02.775354 4026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd37e2bdeeb84f27810d27d79952db82: Generated new TSK 0
I20260809 04:49:02.776054 4026 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:02.780278 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 3941
W20260809 04:49:02.781240 4005 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.174:34931
W20260809 04:49:02.781946 13224 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 3941: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:49:02.782120 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 3941
2026-08-09T04:49:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1867 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:02 dist-test-slave-1jjb krb5kdc[4042](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:02 dist-test-slave-1jjb krb5kdc[4042](info): set up 2 sockets
Aug 09 04:49:02 dist-test-slave-1jjb krb5kdc[4042](info): commencing operation
krb5kdc: starting...
W20260809 04:49:03.863698 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.003s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:49:04 dist-test-slave-1jjb krb5kdc[4042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:49:04.128005 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35245
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:04.445947 4057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:04.446220 4057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:04.446339 4057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:04.452999 4057 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:04.453080 4057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:04.453222 4057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:04.453320 4057 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:04.453408 4057 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:04.462213 4057 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35245
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:49:04.463297 4057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:04.464447 4057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:04.475708 4063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:04.475734 4064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:04.475767 4066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:04.476533 4057 server_base.cc:1034] running on GCE node
I20260809 04:49:04.477212 4057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:04.478461 4057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:04.479676 4057 hybrid_clock.cc:648] HybridClock initialized: now 1786250944479633 us; error 53 us; skew 500 ppm
I20260809 04:49:04.479991 4057 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:49:04 dist-test-slave-1jjb krb5kdc[4042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250944, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:04.483083 4057 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:49:04.484505 4057 webserver.cc:460] Webserver started at http://127.0.0.1:40249/ using document root <none> and password file <none>
I20260809 04:49:04.485193 4057 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:04.485324 4057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:04.485716 4057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:04.488481 4057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "e11b8b78e1464822b0b9669d79e8fe98"
format_stamp: "Formatted at 2026-08-09 04:49:04 on dist-test-slave-1jjb"
server_key: "71f995b646b64eeb870ace84e1002a2c"
server_key_iv: "a126e1ca95aadd681508538343b3d937"
server_key_version: "encryptionkey@0"
I20260809 04:49:04.489230 4057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "e11b8b78e1464822b0b9669d79e8fe98"
format_stamp: "Formatted at 2026-08-09 04:49:04 on dist-test-slave-1jjb"
server_key: "71f995b646b64eeb870ace84e1002a2c"
server_key_iv: "a126e1ca95aadd681508538343b3d937"
server_key_version: "encryptionkey@0"
I20260809 04:49:04.493896 4057 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 04:49:04.496932 4073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:04.498165 4057 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:49:04.498342 4057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "e11b8b78e1464822b0b9669d79e8fe98"
format_stamp: "Formatted at 2026-08-09 04:49:04 on dist-test-slave-1jjb"
server_key: "71f995b646b64eeb870ace84e1002a2c"
server_key_iv: "a126e1ca95aadd681508538343b3d937"
server_key_version: "encryptionkey@0"
I20260809 04:49:04.498494 4057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:04.525038 4057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:04.528962 4057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:04.529256 4057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:04.546759 4057 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35245
I20260809 04:49:04.546800 4128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35245 every 8 connection(s)
I20260809 04:49:04.548010 4057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:04.551499 4129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:04.554019 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 4057
I20260809 04:49:04.554195 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:49:04.554610 13224 external_mini_cluster.cc:1428] Setting key 5bd3bf9c6c9c64c1ad20e4aecb2a0006
I20260809 04:49:04.562327 4129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e11b8b78e1464822b0b9669d79e8fe98: Bootstrap starting.
I20260809 04:49:04.565837 4129 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e11b8b78e1464822b0b9669d79e8fe98: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:04.566720 4129 log.cc:824] T 00000000000000000000000000000000 P e11b8b78e1464822b0b9669d79e8fe98: Log is configured to *not* fsync() on all Append() calls
Aug 09 04:49:04 dist-test-slave-1jjb krb5kdc[4042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 04:49:04.569615 4129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e11b8b78e1464822b0b9669d79e8fe98: No bootstrap required, opened a new log
I20260809 04:49:04.577309 4132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:04.556234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.174:34769 (local address 127.0.0.1:35245)
0809 04:49:04.556970 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:04.556980 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:04.557016 (+ 36us) server_negotiation.cc:407] Connection header received
0809 04:49:04.558447 (+ 1431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:04.558466 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:04.558810 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:04.559097 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:04.560698 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:04.561952 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:04.563541 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:04.564629 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:04.564787 (+ 158us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:04.568608 (+ 3821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:04.568616 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:04.568631 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:04.568669 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:04.572692 (+ 4023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:04.573871 (+ 1179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:04.573879 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:04.573886 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:04.574067 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:04.574510 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:04.574514 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:04.574517 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:04.575137 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:04.575474 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:04.575820 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 04:49:04.576157 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":330,"threads_started":1}
I20260809 04:49:04.579298 4129 raft_consensus.cc:348] T 00000000000000000000000000000000 P e11b8b78e1464822b0b9669d79e8fe98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11b8b78e1464822b0b9669d79e8fe98" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35245 } }
I20260809 04:49:04.579727 4129 raft_consensus.cc:374] T 00000000000000000000000000000000 P e11b8b78e1464822b0b9669d79e8fe98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:04.579845 4129 raft_consensus.cc:729] T 00000000000000000000000000000000 P e11b8b78e1464822b0b9669d79e8fe98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e11b8b78e1464822b0b9669d79e8fe98, State: Initialized, Role: FOLLOWER
I20260809 04:49:04.580454 4129 consensus_queue.cc:260] T 00000000000000000000000000000000 P e11b8b78e1464822b0b9669d79e8fe98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11b8b78e1464822b0b9669d79e8fe98" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35245 } }
I20260809 04:49:04.580726 4129 raft_consensus.cc:388] T 00000000000000000000000000000000 P e11b8b78e1464822b0b9669d79e8fe98 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:04.580803 13224 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:04.580883 4129 raft_consensus.cc:482] T 00000000000000000000000000000000 P e11b8b78e1464822b0b9669d79e8fe98 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:04.581044 4129 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e11b8b78e1464822b0b9669d79e8fe98 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:04.584136 4129 raft_consensus.cc:504] T 00000000000000000000000000000000 P e11b8b78e1464822b0b9669d79e8fe98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11b8b78e1464822b0b9669d79e8fe98" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35245 } }
I20260809 04:49:04.584802 4129 leader_election.cc:302] T 00000000000000000000000000000000 P e11b8b78e1464822b0b9669d79e8fe98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e11b8b78e1464822b0b9669d79e8fe98; no voters:
I20260809 04:49:04.585484 4129 leader_election.cc:288] T 00000000000000000000000000000000 P e11b8b78e1464822b0b9669d79e8fe98 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:04.585526 4138 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e11b8b78e1464822b0b9669d79e8fe98 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:04.587589 4129 sys_catalog.cc:564] T 00000000000000000000000000000000 P e11b8b78e1464822b0b9669d79e8fe98 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:04.587945 4138 raft_consensus.cc:686] T 00000000000000000000000000000000 P e11b8b78e1464822b0b9669d79e8fe98 [term 1 LEADER]: Becoming Leader. State: Replica: e11b8b78e1464822b0b9669d79e8fe98, State: Running, Role: LEADER
I20260809 04:49:04.588466 4138 consensus_queue.cc:237] T 00000000000000000000000000000000 P e11b8b78e1464822b0b9669d79e8fe98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11b8b78e1464822b0b9669d79e8fe98" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35245 } }
I20260809 04:49:04.591131 4139 sys_catalog.cc:455] T 00000000000000000000000000000000 P e11b8b78e1464822b0b9669d79e8fe98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e11b8b78e1464822b0b9669d79e8fe98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11b8b78e1464822b0b9669d79e8fe98" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35245 } } }
I20260809 04:49:04.591498 4139 sys_catalog.cc:458] T 00000000000000000000000000000000 P e11b8b78e1464822b0b9669d79e8fe98 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:04.592000 4140 sys_catalog.cc:455] T 00000000000000000000000000000000 P e11b8b78e1464822b0b9669d79e8fe98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e11b8b78e1464822b0b9669d79e8fe98. Latest consensus state: current_term: 1 leader_uuid: "e11b8b78e1464822b0b9669d79e8fe98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e11b8b78e1464822b0b9669d79e8fe98" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35245 } } }
I20260809 04:49:04.592335 4140 sys_catalog.cc:458] T 00000000000000000000000000000000 P e11b8b78e1464822b0b9669d79e8fe98 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:04.608057 4132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:04.586516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.174:35057 (local address 127.0.0.1:35245)
0809 04:49:04.587207 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:04.587212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:04.587287 (+ 75us) server_negotiation.cc:407] Connection header received
0809 04:49:04.587542 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:04.587546 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:04.587626 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:04.587722 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:04.590063 (+ 2341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:04.591625 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:04.594011 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:04.594782 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:04.594920 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:04.598372 (+ 3452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:04.598380 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:04.598387 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:04.598429 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:04.602883 (+ 4454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:04.604098 (+ 1215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:04.604106 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:04.604112 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:04.604273 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:04.605010 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:04.605018 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:04.605023 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:04.605555 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:04.607170 (+ 1615us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:04.607372 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 04:49:04.607582 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589}
I20260809 04:49:04.614660 4146 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:04.619796 4146 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:04.628062 4146 catalog_manager.cc:1347] Generated new cluster ID: ace92009e7c94d12be54eff2acf6448d
I20260809 04:49:04.628182 4146 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:04.648273 4146 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:04.649442 4146 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:04.656642 4146 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e11b8b78e1464822b0b9669d79e8fe98: Generated new TSK 0
I20260809 04:49:04.657256 4146 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:04.665331 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 4057
I20260809 04:49:04.666621 4124 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:49:04.750499 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 4057
W20260809 04:49:04.771829 4040 connection.cc:535] client connection to 127.0.0.1:35245 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35245): Connection reset by peer (error 104)
2026-08-09T04:49:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1972 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:04 dist-test-slave-1jjb krb5kdc[4163](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:04 dist-test-slave-1jjb krb5kdc[4163](info): set up 2 sockets
Aug 09 04:49:04 dist-test-slave-1jjb krb5kdc[4163](info): commencing operation
krb5kdc: starting...
W20260809 04:49:05.852425 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.002s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:49:06 dist-test-slave-1jjb krb5kdc[4163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:49:06.099148 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39923
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39923
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:06.413568 4178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:06.413822 4178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:06.413913 4178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:06.419946 4178 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:06.420027 4178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:06.420079 4178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:06.420226 4178 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:06.420322 4178 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:06.428635 4178 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39923
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:49:06.429692 4178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:06.430864 4178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:06.441687 4183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:06.441679 4186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:06.441704 4184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:06.442561 4178 server_base.cc:1034] running on GCE node
I20260809 04:49:06.443171 4178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:06.444375 4178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:06.445592 4178 hybrid_clock.cc:648] HybridClock initialized: now 1786250946445556 us; error 47 us; skew 500 ppm
I20260809 04:49:06.445909 4178 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:49:06 dist-test-slave-1jjb krb5kdc[4163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250946, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:06.448787 4178 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:49:06.450098 4178 webserver.cc:460] Webserver started at http://127.0.0.1:46875/ using document root <none> and password file <none>
I20260809 04:49:06.450675 4178 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:06.450855 4178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:06.451149 4178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:06.453761 4178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "cf53f5075af34f40b5183248e3f5a568"
format_stamp: "Formatted at 2026-08-09 04:49:06 on dist-test-slave-1jjb"
server_key: "ab81c788689a0cafd8537d9e6eb19c6b"
server_key_iv: "fd54b9fe1ab2c18de4fd91dd092cf477"
server_key_version: "encryptionkey@0"
I20260809 04:49:06.454440 4178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "cf53f5075af34f40b5183248e3f5a568"
format_stamp: "Formatted at 2026-08-09 04:49:06 on dist-test-slave-1jjb"
server_key: "ab81c788689a0cafd8537d9e6eb19c6b"
server_key_iv: "fd54b9fe1ab2c18de4fd91dd092cf477"
server_key_version: "encryptionkey@0"
I20260809 04:49:06.458897 4178 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 04:49:06.462157 4194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:06.463261 4178 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:49:06.463439 4178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "cf53f5075af34f40b5183248e3f5a568"
format_stamp: "Formatted at 2026-08-09 04:49:06 on dist-test-slave-1jjb"
server_key: "ab81c788689a0cafd8537d9e6eb19c6b"
server_key_iv: "fd54b9fe1ab2c18de4fd91dd092cf477"
server_key_version: "encryptionkey@0"
I20260809 04:49:06.463616 4178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:06.492307 4178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:06.495734 4178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:06.495985 4178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:06.513206 4178 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39923
I20260809 04:49:06.513242 4247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39923 every 8 connection(s)
I20260809 04:49:06.514365 4178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:06.517562 4248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:06.521800 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 4178
I20260809 04:49:06.522023 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:49:06.522524 13224 external_mini_cluster.cc:1428] Setting key 81abeda242b02685f27957b4449bb641
I20260809 04:49:06.527705 4248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf53f5075af34f40b5183248e3f5a568: Bootstrap starting.
I20260809 04:49:06.531123 4248 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf53f5075af34f40b5183248e3f5a568: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:06.532328 4248 log.cc:824] T 00000000000000000000000000000000 P cf53f5075af34f40b5183248e3f5a568: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:06.535105 4248 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf53f5075af34f40b5183248e3f5a568: No bootstrap required, opened a new log
I20260809 04:49:06.535121 4251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:06.524159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.174:54467 (local address 127.0.0.1:39923)
0809 04:49:06.524852 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:06.524861 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:06.524914 (+ 53us) server_negotiation.cc:407] Connection header received
0809 04:49:06.526382 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:06.526401 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:06.526691 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:06.527070 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:06.527645 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:06.528981 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:06.530822 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:06.531537 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:06.531656 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:06.532876 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:06.532890 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:06.532929 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:06.533631 (+ 702us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:06.533735 (+ 104us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:06.533762 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:06.533871 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:06.534171 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 04:49:06.534322 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":321,"threads_started":1}
I20260809 04:49:06.538585 13224 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:06.544209 4248 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf53f5075af34f40b5183248e3f5a568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf53f5075af34f40b5183248e3f5a568" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39923 } }
I20260809 04:49:06.544626 4248 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf53f5075af34f40b5183248e3f5a568 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:06.544721 4248 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf53f5075af34f40b5183248e3f5a568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf53f5075af34f40b5183248e3f5a568, State: Initialized, Role: FOLLOWER
I20260809 04:49:06.545365 4248 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf53f5075af34f40b5183248e3f5a568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf53f5075af34f40b5183248e3f5a568" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39923 } }
I20260809 04:49:06.545651 4248 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf53f5075af34f40b5183248e3f5a568 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:06.545766 4248 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf53f5075af34f40b5183248e3f5a568 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:06.545922 4248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf53f5075af34f40b5183248e3f5a568 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:06.548281 4248 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf53f5075af34f40b5183248e3f5a568 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf53f5075af34f40b5183248e3f5a568" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39923 } }
I20260809 04:49:06.548784 4248 leader_election.cc:302] T 00000000000000000000000000000000 P cf53f5075af34f40b5183248e3f5a568 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf53f5075af34f40b5183248e3f5a568; no voters:
I20260809 04:49:06.549475 4248 leader_election.cc:288] T 00000000000000000000000000000000 P cf53f5075af34f40b5183248e3f5a568 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:06.549615 4259 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf53f5075af34f40b5183248e3f5a568 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:06.550380 4251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:06.543682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.174:43895 (local address 127.0.0.1:39923)
0809 04:49:06.543863 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:06.543870 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:06.544227 (+ 357us) server_negotiation.cc:407] Connection header received
0809 04:49:06.544315 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:06.544319 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:06.544384 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:06.544490 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:06.544974 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:06.546006 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:06.547101 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:06.547776 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:06.547959 (+ 183us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:06.549181 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:06.549186 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:06.549194 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:06.549216 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:06.549274 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:06.549302 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:06.549452 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:06.549755 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 04:49:06.549814 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 04:49:06.550384 4259 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf53f5075af34f40b5183248e3f5a568 [term 1 LEADER]: Becoming Leader. State: Replica: cf53f5075af34f40b5183248e3f5a568, State: Running, Role: LEADER
I20260809 04:49:06.551609 4259 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf53f5075af34f40b5183248e3f5a568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf53f5075af34f40b5183248e3f5a568" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39923 } }
I20260809 04:49:06.552438 4248 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf53f5075af34f40b5183248e3f5a568 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:06.554332 4261 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf53f5075af34f40b5183248e3f5a568 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf53f5075af34f40b5183248e3f5a568. Latest consensus state: current_term: 1 leader_uuid: "cf53f5075af34f40b5183248e3f5a568" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf53f5075af34f40b5183248e3f5a568" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39923 } } }
I20260809 04:49:06.555297 4261 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf53f5075af34f40b5183248e3f5a568 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:06.554965 4262 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf53f5075af34f40b5183248e3f5a568 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf53f5075af34f40b5183248e3f5a568" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf53f5075af34f40b5183248e3f5a568" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39923 } } }
I20260809 04:49:06.555779 4262 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf53f5075af34f40b5183248e3f5a568 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:06.556228 4268 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:06.561267 4268 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:06.569343 4268 catalog_manager.cc:1347] Generated new cluster ID: 6d198aae63904df5ae46b00bce9a1b59
I20260809 04:49:06.569515 4268 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:06.608163 4268 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:06.609390 4268 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:06.617416 4268 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf53f5075af34f40b5183248e3f5a568: Generated new TSK 0
I20260809 04:49:06.618050 4268 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:06.622509 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 4178
W20260809 04:49:06.623649 4243 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.174:54467
W20260809 04:49:06.624411 13224 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 4178: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:49:06.624569 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 4178
W20260809 04:49:06.645921 4161 connection.cc:535] client connection to 127.0.0.1:39923 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39923): Connection reset by peer (error 104)
2026-08-09T04:49:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1869 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:06 dist-test-slave-1jjb krb5kdc[4281](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:06 dist-test-slave-1jjb krb5kdc[4281](info): set up 2 sockets
Aug 09 04:49:06 dist-test-slave-1jjb krb5kdc[4281](info): commencing operation
krb5kdc: starting...
W20260809 04:49:07.699590 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.002s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:49:07 dist-test-slave-1jjb krb5kdc[4281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 04:49:07.954592 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35441
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:08.259591 4296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:08.259831 4296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:08.259925 4296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:08.266466 4296 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:08.266546 4296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:08.266585 4296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:08.266630 4296 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:08.266661 4296 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:08.275153 4296 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35441
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250603326224-13224-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:49:08.276160 4296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:08.277331 4296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:08.288317 4302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:08.288405 4301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:08.288365 4304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:08.289252 4296 server_base.cc:1034] running on GCE node
I20260809 04:49:08.289954 4296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:08.291296 4296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:08.292526 4296 hybrid_clock.cc:648] HybridClock initialized: now 1786250948292489 us; error 45 us; skew 500 ppm
I20260809 04:49:08.292845 4296 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:49:08 dist-test-slave-1jjb krb5kdc[4281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250948, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:08.300884 4296 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 04:49:08.302359 4296 webserver.cc:460] Webserver started at http://127.0.0.1:32811/ using document root <none> and password file <none>
I20260809 04:49:08.302996 4296 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:08.303133 4296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:08.303452 4296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:08.306150 4296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "cfb8fd947d994a84aed8ffe49c5c40ea"
format_stamp: "Formatted at 2026-08-09 04:49:08 on dist-test-slave-1jjb"
server_key: "c94614c2d95b6b3109d2697709ab1610"
server_key_iv: "e215161f67518fbc24a19960658cb980"
server_key_version: "encryptionkey@0"
I20260809 04:49:08.306841 4296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "cfb8fd947d994a84aed8ffe49c5c40ea"
format_stamp: "Formatted at 2026-08-09 04:49:08 on dist-test-slave-1jjb"
server_key: "c94614c2d95b6b3109d2697709ab1610"
server_key_iv: "e215161f67518fbc24a19960658cb980"
server_key_version: "encryptionkey@0"
I20260809 04:49:08.311517 4296 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 04:49:08.314707 4311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:08.315886 4296 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:49:08.316036 4296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "cfb8fd947d994a84aed8ffe49c5c40ea"
format_stamp: "Formatted at 2026-08-09 04:49:08 on dist-test-slave-1jjb"
server_key: "c94614c2d95b6b3109d2697709ab1610"
server_key_iv: "e215161f67518fbc24a19960658cb980"
server_key_version: "encryptionkey@0"
I20260809 04:49:08.316280 4296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:08.321451 4296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:08.325346 4296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:08.325618 4296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:08.342918 4296 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35441
I20260809 04:49:08.342983 4364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35441 every 8 connection(s)
I20260809 04:49:08.344069 4296 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:08.346549 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 4296
I20260809 04:49:08.346717 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:49:08.347211 13224 external_mini_cluster.cc:1428] Setting key e36c3ee8f371411b23f8435d23813c3a
I20260809 04:49:08.347738 4365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:08.354207 4369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:08.348799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.174:33367 (local address 127.0.0.1:35441)
0809 04:49:08.349542 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:08.349552 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:08.349589 (+ 37us) server_negotiation.cc:407] Connection header received
0809 04:49:08.351007 (+ 1418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:08.351027 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:08.351329 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:08.351707 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:08.352722 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:08.352727 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:08.352738 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:08.352750 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:08.352806 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:08.352825 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:08.353074 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:08.353206 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 04:49:08.353385 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":355,"threads_started":1}
I20260809 04:49:08.357559 13224 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:08.360284 4365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfb8fd947d994a84aed8ffe49c5c40ea: Bootstrap starting.
I20260809 04:49:08.363700 4365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfb8fd947d994a84aed8ffe49c5c40ea: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:08.364157 4369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:08.361915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.174:39745 (local address 127.0.0.1:35441)
0809 04:49:08.362108 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:08.362112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:08.362411 (+ 299us) server_negotiation.cc:407] Connection header received
0809 04:49:08.362516 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:08.362520 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:08.362574 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:08.362669 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:08.363448 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:08.363454 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:08.363461 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:08.363480 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:08.363523 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:08.363544 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:08.363661 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:08.363768 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 04:49:08.363837 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:49:08.364912 4365 log.cc:824] T 00000000000000000000000000000000 P cfb8fd947d994a84aed8ffe49c5c40ea: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:08.367242 4365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfb8fd947d994a84aed8ffe49c5c40ea: No bootstrap required, opened a new log
I20260809 04:49:08.373948 4365 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfb8fd947d994a84aed8ffe49c5c40ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb8fd947d994a84aed8ffe49c5c40ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35441 } }
I20260809 04:49:08.374291 4365 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfb8fd947d994a84aed8ffe49c5c40ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:08.374399 4365 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfb8fd947d994a84aed8ffe49c5c40ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfb8fd947d994a84aed8ffe49c5c40ea, State: Initialized, Role: FOLLOWER
I20260809 04:49:08.374909 4365 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfb8fd947d994a84aed8ffe49c5c40ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb8fd947d994a84aed8ffe49c5c40ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35441 } }
I20260809 04:49:08.375144 4365 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfb8fd947d994a84aed8ffe49c5c40ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:08.375293 4365 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfb8fd947d994a84aed8ffe49c5c40ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:08.375456 4365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfb8fd947d994a84aed8ffe49c5c40ea [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:08.377398 4365 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfb8fd947d994a84aed8ffe49c5c40ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb8fd947d994a84aed8ffe49c5c40ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35441 } }
I20260809 04:49:08.377842 4365 leader_election.cc:302] T 00000000000000000000000000000000 P cfb8fd947d994a84aed8ffe49c5c40ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfb8fd947d994a84aed8ffe49c5c40ea; no voters:
I20260809 04:49:08.378408 4365 leader_election.cc:288] T 00000000000000000000000000000000 P cfb8fd947d994a84aed8ffe49c5c40ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:08.378487 4377 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfb8fd947d994a84aed8ffe49c5c40ea [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:08.379101 4377 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfb8fd947d994a84aed8ffe49c5c40ea [term 1 LEADER]: Becoming Leader. State: Replica: cfb8fd947d994a84aed8ffe49c5c40ea, State: Running, Role: LEADER
I20260809 04:49:08.379585 4377 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfb8fd947d994a84aed8ffe49c5c40ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb8fd947d994a84aed8ffe49c5c40ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35441 } }
I20260809 04:49:08.380174 4365 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfb8fd947d994a84aed8ffe49c5c40ea [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:08.382241 4379 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfb8fd947d994a84aed8ffe49c5c40ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfb8fd947d994a84aed8ffe49c5c40ea. Latest consensus state: current_term: 1 leader_uuid: "cfb8fd947d994a84aed8ffe49c5c40ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb8fd947d994a84aed8ffe49c5c40ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35441 } } }
I20260809 04:49:08.382213 4378 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfb8fd947d994a84aed8ffe49c5c40ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfb8fd947d994a84aed8ffe49c5c40ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfb8fd947d994a84aed8ffe49c5c40ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35441 } } }
I20260809 04:49:08.382845 4379 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfb8fd947d994a84aed8ffe49c5c40ea [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:08.382853 4378 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfb8fd947d994a84aed8ffe49c5c40ea [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:08.383797 4387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:08.388388 4387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:08.395948 4387 catalog_manager.cc:1347] Generated new cluster ID: 2d53dbaffc244473ba993c63338a0db4
I20260809 04:49:08.396046 4387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:08.406854 4387 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:08.407207 4387 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:08.412896 4387 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfb8fd947d994a84aed8ffe49c5c40ea: Generated new TSK 0
I20260809 04:49:08.413551 4387 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:08.419797 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 4296
W20260809 04:49:08.420863 4360 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.174:33367
W20260809 04:49:08.421559 13224 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 4296: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:49:08.421705 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 4296
2026-08-09T04:49:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1796 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:08 dist-test-slave-1jjb krb5kdc[4401](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:08 dist-test-slave-1jjb krb5kdc[4401](info): set up 2 sockets
Aug 09 04:49:08 dist-test-slave-1jjb krb5kdc[4401](info): commencing operation
krb5kdc: starting...
W20260809 04:49:09.521319 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.002s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:49:09 dist-test-slave-1jjb krb5kdc[4401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:49:09.791442 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:43523
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:39579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:43523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:10.099478 4414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:10.099722 4414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:10.099804 4414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:10.106482 4414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:10.106580 4414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:10.106624 4414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:10.106690 4414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:10.106736 4414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:10.115584 4414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:39579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:43523
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:43523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:49:10.116613 4414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:10.117723 4414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:10.128947 4419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:10.128916 4420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:10.128892 4422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:10.129760 4414 server_base.cc:1034] running on GCE node
I20260809 04:49:10.130430 4414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:10.131660 4414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:10.132848 4414 hybrid_clock.cc:648] HybridClock initialized: now 1786250950132800 us; error 40 us; skew 500 ppm
I20260809 04:49:10.133186 4414 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:49:10 dist-test-slave-1jjb krb5kdc[4401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250950, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:10.136009 4414 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:49:10.137478 4414 webserver.cc:460] Webserver started at http://127.12.234.62:40465/ using document root <none> and password file <none>
I20260809 04:49:10.138077 4414 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:10.138195 4414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:10.138476 4414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:10.141253 4414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "7370c9f8ee27473db584c4bcf023efa9"
format_stamp: "Formatted at 2026-08-09 04:49:10 on dist-test-slave-1jjb"
server_key: "a22ebb1e4a2afde9e142d0081eb91ef2"
server_key_iv: "0399ccd94223ab23f8dd0229bf9140b2"
server_key_version: "encryptionkey@0"
I20260809 04:49:10.141999 4414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "7370c9f8ee27473db584c4bcf023efa9"
format_stamp: "Formatted at 2026-08-09 04:49:10 on dist-test-slave-1jjb"
server_key: "a22ebb1e4a2afde9e142d0081eb91ef2"
server_key_iv: "0399ccd94223ab23f8dd0229bf9140b2"
server_key_version: "encryptionkey@0"
I20260809 04:49:10.146836 4414 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260809 04:49:10.150031 4429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:10.151223 4414 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 04:49:10.151415 4414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "7370c9f8ee27473db584c4bcf023efa9"
format_stamp: "Formatted at 2026-08-09 04:49:10 on dist-test-slave-1jjb"
server_key: "a22ebb1e4a2afde9e142d0081eb91ef2"
server_key_iv: "0399ccd94223ab23f8dd0229bf9140b2"
server_key_version: "encryptionkey@0"
I20260809 04:49:10.151607 4414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:10.161553 4414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:10.165148 4414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:10.165395 4414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:10.183187 4414 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:43523
I20260809 04:49:10.183264 4483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:43523 every 8 connection(s)
I20260809 04:49:10.184522 4414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:10.188179 4484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:10.194031 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 4414
I20260809 04:49:10.194245 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:49:10.194824 13224 external_mini_cluster.cc:1428] Setting key 880491346000d7c3cb68fa22349334d8
I20260809 04:49:10.199002 4484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7370c9f8ee27473db584c4bcf023efa9: Bootstrap starting.
I20260809 04:49:10.202428 4484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7370c9f8ee27473db584c4bcf023efa9: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:10.203541 4484 log.cc:824] T 00000000000000000000000000000000 P 7370c9f8ee27473db584c4bcf023efa9: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:10.206501 4484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7370c9f8ee27473db584c4bcf023efa9: No bootstrap required, opened a new log
Aug 09 04:49:10 dist-test-slave-1jjb krb5kdc[4401](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:49:10.214507 4484 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7370c9f8ee27473db584c4bcf023efa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7370c9f8ee27473db584c4bcf023efa9" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 43523 } }
I20260809 04:49:10.214867 4484 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7370c9f8ee27473db584c4bcf023efa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:10.214998 4484 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7370c9f8ee27473db584c4bcf023efa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7370c9f8ee27473db584c4bcf023efa9, State: Initialized, Role: FOLLOWER
I20260809 04:49:10.215639 4484 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7370c9f8ee27473db584c4bcf023efa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7370c9f8ee27473db584c4bcf023efa9" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 43523 } }
I20260809 04:49:10.215927 4484 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7370c9f8ee27473db584c4bcf023efa9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:10.216066 4484 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7370c9f8ee27473db584c4bcf023efa9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:10.216358 4484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7370c9f8ee27473db584c4bcf023efa9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:10.218415 4487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:10.196586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51477 (local address 127.12.234.62:43523)
0809 04:49:10.197432 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:10.197442 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:10.197480 (+ 38us) server_negotiation.cc:407] Connection header received
0809 04:49:10.199047 (+ 1567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:10.199095 (+ 48us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:10.199490 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:10.199771 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:10.201503 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:10.202962 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:10.204622 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:10.205301 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:10.205426 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:10.210366 (+ 4940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:10.210373 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:10.210385 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:10.210422 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:10.213944 (+ 3522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:10.214766 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:10.214773 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:10.214778 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:10.215019 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:10.215542 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:10.215545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:10.215547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:10.216209 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:10.216593 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:10.216950 (+ 357us) server_negotiation.cc:299] Negotiation successful
0809 04:49:10.217307 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":436,"threads_started":1}
I20260809 04:49:10.218914 4484 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7370c9f8ee27473db584c4bcf023efa9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7370c9f8ee27473db584c4bcf023efa9" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 43523 } }
I20260809 04:49:10.219393 4484 leader_election.cc:302] T 00000000000000000000000000000000 P 7370c9f8ee27473db584c4bcf023efa9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7370c9f8ee27473db584c4bcf023efa9; no voters:
I20260809 04:49:10.220037 4484 leader_election.cc:288] T 00000000000000000000000000000000 P 7370c9f8ee27473db584c4bcf023efa9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:10.220145 4489 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7370c9f8ee27473db584c4bcf023efa9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:10.222515 13224 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:10.222481 4484 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7370c9f8ee27473db584c4bcf023efa9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:10.227156 4489 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7370c9f8ee27473db584c4bcf023efa9 [term 1 LEADER]: Becoming Leader. State: Replica: 7370c9f8ee27473db584c4bcf023efa9, State: Running, Role: LEADER
I20260809 04:49:10.227767 4489 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7370c9f8ee27473db584c4bcf023efa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7370c9f8ee27473db584c4bcf023efa9" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 43523 } }
I20260809 04:49:10.230962 4490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7370c9f8ee27473db584c4bcf023efa9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7370c9f8ee27473db584c4bcf023efa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7370c9f8ee27473db584c4bcf023efa9" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 43523 } } }
I20260809 04:49:10.231437 4490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7370c9f8ee27473db584c4bcf023efa9 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:10.232004 4491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7370c9f8ee27473db584c4bcf023efa9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7370c9f8ee27473db584c4bcf023efa9. Latest consensus state: current_term: 1 leader_uuid: "7370c9f8ee27473db584c4bcf023efa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7370c9f8ee27473db584c4bcf023efa9" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 43523 } } }
I20260809 04:49:10.232328 4491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7370c9f8ee27473db584c4bcf023efa9 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:10.234362 4498 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:10.240139 4498 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:10.258124 4498 catalog_manager.cc:1347] Generated new cluster ID: f49bc3f05a5a4244adfba6fc2e174906
I20260809 04:49:10.258214 4498 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:10.280059 4498 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:10.281858 4487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:10.248544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60111 (local address 127.12.234.62:43523)
0809 04:49:10.248737 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:10.248746 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:10.255615 (+ 6869us) server_negotiation.cc:407] Connection header received
0809 04:49:10.255973 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:10.255982 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:10.256098 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:10.256194 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:10.259348 (+ 3154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:10.261016 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:10.263759 (+ 2743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:10.264879 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:10.265020 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:10.269729 (+ 4709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:10.269739 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:10.269747 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:10.269799 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:10.274587 (+ 4788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:10.275840 (+ 1253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:10.275849 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:10.275854 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:10.276101 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:10.276795 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:10.276802 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:10.276808 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:10.277298 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:10.277587 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:10.278401 (+ 814us) server_negotiation.cc:299] Negotiation successful
0809 04:49:10.278644 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":53}
I20260809 04:49:10.282055 4498 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:10.289649 4498 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7370c9f8ee27473db584c4bcf023efa9: Generated new TSK 0
I20260809 04:49:10.290530 4498 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:10.301993 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 4414
I20260809 04:49:10.303368 4479 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:49:10.389916 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 4414
W20260809 04:49:10.411789 4399 connection.cc:535] client connection to 127.12.234.62:43523 recv error: Network error: failed to read from TLS socket (remote: 127.12.234.62:43523): Connection reset by peer (error 104)
2026-08-09T04:49:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1970 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:10 dist-test-slave-1jjb krb5kdc[4518](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:10 dist-test-slave-1jjb krb5kdc[4518](info): set up 2 sockets
Aug 09 04:49:10 dist-test-slave-1jjb krb5kdc[4518](info): commencing operation
krb5kdc: starting...
W20260809 04:49:11.485275 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.001s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:49:11 dist-test-slave-1jjb krb5kdc[4518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:49:11.742632 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:36701
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:39393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:36701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:12.048427 4531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:12.048697 4531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:12.048846 4531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:12.055369 4531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:12.055491 4531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:12.055579 4531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:12.055671 4531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:12.055743 4531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:12.064456 4531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:39393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:36701
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:36701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:49:12.065565 4531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:12.066656 4531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:12.077636 4538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:12.077703 4541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:12.077674 4539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:12.078812 4531 server_base.cc:1034] running on GCE node
I20260809 04:49:12.079481 4531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:12.080751 4531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:12.081943 4531 hybrid_clock.cc:648] HybridClock initialized: now 1786250952081908 us; error 27 us; skew 500 ppm
I20260809 04:49:12.082274 4531 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:49:12 dist-test-slave-1jjb krb5kdc[4518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250952, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:12.085405 4531 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:49:12.086750 4531 webserver.cc:460] Webserver started at http://127.12.234.62:40871/ using document root <none> and password file <none>
I20260809 04:49:12.087348 4531 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:12.087455 4531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:12.087693 4531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:12.090744 4531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "2a6df955262a4ec6a37c5693634b7f0d"
format_stamp: "Formatted at 2026-08-09 04:49:12 on dist-test-slave-1jjb"
server_key: "da445e5f99505a7a160c45d599e06810"
server_key_iv: "3f2d23da03fe5d4fc1c27e0ea081f87c"
server_key_version: "encryptionkey@0"
I20260809 04:49:12.091516 4531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "2a6df955262a4ec6a37c5693634b7f0d"
format_stamp: "Formatted at 2026-08-09 04:49:12 on dist-test-slave-1jjb"
server_key: "da445e5f99505a7a160c45d599e06810"
server_key_iv: "3f2d23da03fe5d4fc1c27e0ea081f87c"
server_key_version: "encryptionkey@0"
I20260809 04:49:12.096705 4531 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 04:49:12.099835 4549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:12.101140 4531 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:49:12.101387 4531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "2a6df955262a4ec6a37c5693634b7f0d"
format_stamp: "Formatted at 2026-08-09 04:49:12 on dist-test-slave-1jjb"
server_key: "da445e5f99505a7a160c45d599e06810"
server_key_iv: "3f2d23da03fe5d4fc1c27e0ea081f87c"
server_key_version: "encryptionkey@0"
I20260809 04:49:12.101574 4531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:12.120034 4531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:12.123785 4531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:12.124058 4531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:12.143309 4531 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:36701
I20260809 04:49:12.143316 4601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:36701 every 8 connection(s)
I20260809 04:49:12.144596 4531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:12.147893 4603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:12.154675 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 4531
I20260809 04:49:12.154861 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:49:12.155282 13224 external_mini_cluster.cc:1428] Setting key f06e7475b37a70503c266fffb3ca423a
I20260809 04:49:12.158031 4603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a6df955262a4ec6a37c5693634b7f0d: Bootstrap starting.
I20260809 04:49:12.161863 4603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a6df955262a4ec6a37c5693634b7f0d: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:12.163120 4603 log.cc:824] T 00000000000000000000000000000000 P 2a6df955262a4ec6a37c5693634b7f0d: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:12.165932 4603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a6df955262a4ec6a37c5693634b7f0d: No bootstrap required, opened a new log
I20260809 04:49:12.169814 4606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:12.157028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58287 (local address 127.12.234.62:36701)
0809 04:49:12.157778 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:12.157792 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:12.157850 (+ 58us) server_negotiation.cc:407] Connection header received
0809 04:49:12.160277 (+ 2427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:12.160301 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:12.160714 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:12.161111 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:12.161584 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.163610 (+ 2026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.165126 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.166121 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.166287 (+ 166us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:12.167304 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:12.167312 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:12.167326 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:12.167942 (+ 616us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:12.168021 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:12.168053 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:12.168219 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:12.168443 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 04:49:12.168619 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":308,"threads_started":1}
I20260809 04:49:12.173595 13224 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:12.175130 4603 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a6df955262a4ec6a37c5693634b7f0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a6df955262a4ec6a37c5693634b7f0d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36701 } }
I20260809 04:49:12.175505 4603 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a6df955262a4ec6a37c5693634b7f0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:12.175597 4603 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a6df955262a4ec6a37c5693634b7f0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a6df955262a4ec6a37c5693634b7f0d, State: Initialized, Role: FOLLOWER
I20260809 04:49:12.176159 4603 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a6df955262a4ec6a37c5693634b7f0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a6df955262a4ec6a37c5693634b7f0d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36701 } }
I20260809 04:49:12.176416 4603 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a6df955262a4ec6a37c5693634b7f0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:12.176573 4603 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a6df955262a4ec6a37c5693634b7f0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:12.176735 4603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a6df955262a4ec6a37c5693634b7f0d [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:12.179175 4603 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a6df955262a4ec6a37c5693634b7f0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a6df955262a4ec6a37c5693634b7f0d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36701 } }
I20260809 04:49:12.179648 4603 leader_election.cc:302] T 00000000000000000000000000000000 P 2a6df955262a4ec6a37c5693634b7f0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a6df955262a4ec6a37c5693634b7f0d; no voters:
I20260809 04:49:12.180397 4603 leader_election.cc:288] T 00000000000000000000000000000000 P 2a6df955262a4ec6a37c5693634b7f0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:12.180457 4613 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a6df955262a4ec6a37c5693634b7f0d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:12.181773 4613 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a6df955262a4ec6a37c5693634b7f0d [term 1 LEADER]: Becoming Leader. State: Replica: 2a6df955262a4ec6a37c5693634b7f0d, State: Running, Role: LEADER
I20260809 04:49:12.182401 4613 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a6df955262a4ec6a37c5693634b7f0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a6df955262a4ec6a37c5693634b7f0d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36701 } }
I20260809 04:49:12.182669 4603 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a6df955262a4ec6a37c5693634b7f0d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:12.185560 4606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:12.178343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40759 (local address 127.12.234.62:36701)
0809 04:49:12.178551 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:12.178558 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:12.179051 (+ 493us) server_negotiation.cc:407] Connection header received
0809 04:49:12.179151 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:12.179157 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:12.179226 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:12.179334 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:12.179801 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.180771 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.182580 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:12.183130 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:12.183290 (+ 160us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:12.184558 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:12.184563 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:12.184567 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:12.184585 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:12.184652 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:12.184680 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:12.184792 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:12.185210 (+ 418us) server_negotiation.cc:299] Negotiation successful
0809 04:49:12.185259 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:49:12.185510 4616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a6df955262a4ec6a37c5693634b7f0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a6df955262a4ec6a37c5693634b7f0d. Latest consensus state: current_term: 1 leader_uuid: "2a6df955262a4ec6a37c5693634b7f0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a6df955262a4ec6a37c5693634b7f0d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36701 } } }
I20260809 04:49:12.185948 4616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a6df955262a4ec6a37c5693634b7f0d [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:12.188390 4615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a6df955262a4ec6a37c5693634b7f0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a6df955262a4ec6a37c5693634b7f0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a6df955262a4ec6a37c5693634b7f0d" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 36701 } } }
I20260809 04:49:12.188995 4615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a6df955262a4ec6a37c5693634b7f0d [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:12.197701 4621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:12.202587 4621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:12.211242 4621 catalog_manager.cc:1347] Generated new cluster ID: eb8c423b9f1f4ba7851aecf5643cd1f8
I20260809 04:49:12.211402 4621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:12.221230 4621 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:12.222697 4621 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:12.232064 4621 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a6df955262a4ec6a37c5693634b7f0d: Generated new TSK 0
I20260809 04:49:12.232929 4621 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:12.247144 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 4531
W20260809 04:49:12.248180 4597 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58287
W20260809 04:49:12.248994 13224 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 4531: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:49:12.249183 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 4531
W20260809 04:49:12.270124 4515 connection.cc:535] client connection to 127.12.234.62:36701 recv error: Network error: failed to read from TLS socket (remote: 127.12.234.62:36701): Connection reset by peer (error 104)
2026-08-09T04:49:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1857 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:12 dist-test-slave-1jjb krb5kdc[4636](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:12 dist-test-slave-1jjb krb5kdc[4636](info): set up 2 sockets
Aug 09 04:49:12 dist-test-slave-1jjb krb5kdc[4636](info): commencing operation
krb5kdc: starting...
W20260809 04:49:13.333055 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.001s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:49:13 dist-test-slave-1jjb krb5kdc[4636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:49:13.598932 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:33219
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:35417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:33219
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:13.913208 4649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:13.913470 4649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:13.913620 4649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:13.919806 4649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:13.919934 4649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:13.920030 4649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:13.920150 4649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:13.920229 4649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:13.929071 4649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:35417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:33219
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:33219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:49:13.930166 4649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:13.931290 4649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:13.942361 4655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:13.942437 4657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:13.943009 4649 server_base.cc:1034] running on GCE node
W20260809 04:49:13.943058 4654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:13.943799 4649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:13.945025 4649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:13.946245 4649 hybrid_clock.cc:648] HybridClock initialized: now 1786250953946216 us; error 35 us; skew 500 ppm
I20260809 04:49:13.946563 4649 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:49:13 dist-test-slave-1jjb krb5kdc[4636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250953, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:13.954430 4649 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:49:13.955696 4649 webserver.cc:460] Webserver started at http://127.12.234.62:38835/ using document root <none> and password file <none>
I20260809 04:49:13.956350 4649 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:13.956481 4649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:13.956738 4649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:13.959344 4649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "f01bb01c6eb34c8ca9445497ec7d6153"
format_stamp: "Formatted at 2026-08-09 04:49:13 on dist-test-slave-1jjb"
server_key: "af95aa1a6dcba09a61db4a1fb45037c7"
server_key_iv: "953ed25039c34e3f9814c72b3352966f"
server_key_version: "encryptionkey@0"
I20260809 04:49:13.960184 4649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "f01bb01c6eb34c8ca9445497ec7d6153"
format_stamp: "Formatted at 2026-08-09 04:49:13 on dist-test-slave-1jjb"
server_key: "af95aa1a6dcba09a61db4a1fb45037c7"
server_key_iv: "953ed25039c34e3f9814c72b3352966f"
server_key_version: "encryptionkey@0"
I20260809 04:49:13.964771 4649 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 04:49:13.967633 4664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:13.968808 4649 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:49:13.969017 4649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "f01bb01c6eb34c8ca9445497ec7d6153"
format_stamp: "Formatted at 2026-08-09 04:49:13 on dist-test-slave-1jjb"
server_key: "af95aa1a6dcba09a61db4a1fb45037c7"
server_key_iv: "953ed25039c34e3f9814c72b3352966f"
server_key_version: "encryptionkey@0"
I20260809 04:49:13.969234 4649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:13.974543 4649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:13.978036 4649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:13.978417 4649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:13.996116 4649 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:33219
I20260809 04:49:13.996213 4721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:33219 every 8 connection(s)
I20260809 04:49:13.997274 4649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:14.000438 4722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:14.002780 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 4649
I20260809 04:49:14.002990 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:49:14.003522 13224 external_mini_cluster.cc:1428] Setting key 85bf803047e18ab04bf160359e7a1ded
I20260809 04:49:14.010824 4725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:14.005245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50935 (local address 127.12.234.62:33219)
0809 04:49:14.005975 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:14.005987 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:14.006024 (+ 37us) server_negotiation.cc:407] Connection header received
0809 04:49:14.007572 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:14.007593 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:14.007956 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:14.008320 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:14.009345 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:14.009351 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:14.009379 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:14.009400 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:14.009456 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:14.009476 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:14.009648 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:14.009753 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 04:49:14.009944 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":360,"threads_started":1}
I20260809 04:49:14.012065 4722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f01bb01c6eb34c8ca9445497ec7d6153: Bootstrap starting.
I20260809 04:49:14.014302 13224 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:14.015200 4722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f01bb01c6eb34c8ca9445497ec7d6153: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:14.016412 4722 log.cc:824] T 00000000000000000000000000000000 P f01bb01c6eb34c8ca9445497ec7d6153: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:14.019052 4722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f01bb01c6eb34c8ca9445497ec7d6153: No bootstrap required, opened a new log
I20260809 04:49:14.022254 4725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:14.019523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47927 (local address 127.12.234.62:33219)
0809 04:49:14.019717 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:14.019722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:14.019969 (+ 247us) server_negotiation.cc:407] Connection header received
0809 04:49:14.020200 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:14.020207 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:14.020277 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:14.020410 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:14.021416 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:14.021421 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:14.021424 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:14.021441 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:14.021483 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:14.021504 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:14.021664 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:14.021825 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 04:49:14.021902 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 04:49:14.027235 4722 raft_consensus.cc:348] T 00000000000000000000000000000000 P f01bb01c6eb34c8ca9445497ec7d6153 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f01bb01c6eb34c8ca9445497ec7d6153" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 33219 } }
I20260809 04:49:14.027549 4722 raft_consensus.cc:374] T 00000000000000000000000000000000 P f01bb01c6eb34c8ca9445497ec7d6153 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:14.027635 4722 raft_consensus.cc:729] T 00000000000000000000000000000000 P f01bb01c6eb34c8ca9445497ec7d6153 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f01bb01c6eb34c8ca9445497ec7d6153, State: Initialized, Role: FOLLOWER
I20260809 04:49:14.028275 4722 consensus_queue.cc:260] T 00000000000000000000000000000000 P f01bb01c6eb34c8ca9445497ec7d6153 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f01bb01c6eb34c8ca9445497ec7d6153" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 33219 } }
I20260809 04:49:14.028514 4722 raft_consensus.cc:388] T 00000000000000000000000000000000 P f01bb01c6eb34c8ca9445497ec7d6153 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:14.028663 4722 raft_consensus.cc:482] T 00000000000000000000000000000000 P f01bb01c6eb34c8ca9445497ec7d6153 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:14.028832 4722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f01bb01c6eb34c8ca9445497ec7d6153 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:14.031083 4722 raft_consensus.cc:504] T 00000000000000000000000000000000 P f01bb01c6eb34c8ca9445497ec7d6153 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f01bb01c6eb34c8ca9445497ec7d6153" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 33219 } }
I20260809 04:49:14.031672 4722 leader_election.cc:302] T 00000000000000000000000000000000 P f01bb01c6eb34c8ca9445497ec7d6153 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f01bb01c6eb34c8ca9445497ec7d6153; no voters:
I20260809 04:49:14.032380 4722 leader_election.cc:288] T 00000000000000000000000000000000 P f01bb01c6eb34c8ca9445497ec7d6153 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:14.032476 4732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f01bb01c6eb34c8ca9445497ec7d6153 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:14.033126 4732 raft_consensus.cc:686] T 00000000000000000000000000000000 P f01bb01c6eb34c8ca9445497ec7d6153 [term 1 LEADER]: Becoming Leader. State: Replica: f01bb01c6eb34c8ca9445497ec7d6153, State: Running, Role: LEADER
I20260809 04:49:14.033684 4732 consensus_queue.cc:237] T 00000000000000000000000000000000 P f01bb01c6eb34c8ca9445497ec7d6153 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f01bb01c6eb34c8ca9445497ec7d6153" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 33219 } }
I20260809 04:49:14.034497 4722 sys_catalog.cc:564] T 00000000000000000000000000000000 P f01bb01c6eb34c8ca9445497ec7d6153 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:14.036813 4734 sys_catalog.cc:455] T 00000000000000000000000000000000 P f01bb01c6eb34c8ca9445497ec7d6153 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f01bb01c6eb34c8ca9445497ec7d6153. Latest consensus state: current_term: 1 leader_uuid: "f01bb01c6eb34c8ca9445497ec7d6153" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f01bb01c6eb34c8ca9445497ec7d6153" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 33219 } } }
I20260809 04:49:14.037216 4734 sys_catalog.cc:458] T 00000000000000000000000000000000 P f01bb01c6eb34c8ca9445497ec7d6153 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:14.037017 4733 sys_catalog.cc:455] T 00000000000000000000000000000000 P f01bb01c6eb34c8ca9445497ec7d6153 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f01bb01c6eb34c8ca9445497ec7d6153" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f01bb01c6eb34c8ca9445497ec7d6153" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 33219 } } }
I20260809 04:49:14.037379 4733 sys_catalog.cc:458] T 00000000000000000000000000000000 P f01bb01c6eb34c8ca9445497ec7d6153 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:14.041426 4740 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:14.047441 4740 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:14.055397 4740 catalog_manager.cc:1347] Generated new cluster ID: c5cef6ac61ac4b9a856a7131b8d93045
I20260809 04:49:14.055485 4740 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:14.104306 4740 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:14.104764 4740 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:14.114375 4740 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f01bb01c6eb34c8ca9445497ec7d6153: Generated new TSK 0
I20260809 04:49:14.115195 4740 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:14.122215 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 4649
W20260809 04:49:14.123232 4716 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50935
W20260809 04:49:14.123814 13224 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 4649: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:49:14.123975 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 4649
2026-08-09T04:49:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1874 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:14 dist-test-slave-1jjb krb5kdc[4754](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:14 dist-test-slave-1jjb krb5kdc[4754](info): set up 2 sockets
Aug 09 04:49:14 dist-test-slave-1jjb krb5kdc[4754](info): commencing operation
krb5kdc: starting...
W20260809 04:49:15.227726 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:49:15 dist-test-slave-1jjb krb5kdc[4754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:49:15.511150 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:46025
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:44877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:46025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:15.821417 4767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:15.821686 4767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:15.821842 4767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:15.828547 4767 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:15.828634 4767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:15.828743 4767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:15.828845 4767 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:15.828925 4767 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:15.838292 4767 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:44877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:46025
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:46025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:49:15.839723 4767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:15.840962 4767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:15.851864 4776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:15.851979 4773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:15.852034 4774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:15.853173 4767 server_base.cc:1034] running on GCE node
I20260809 04:49:15.853839 4767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:15.855085 4767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:15.856280 4767 hybrid_clock.cc:648] HybridClock initialized: now 1786250955856226 us; error 42 us; skew 500 ppm
I20260809 04:49:15.856593 4767 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:49:15 dist-test-slave-1jjb krb5kdc[4754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250955, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:15.859567 4767 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:49:15.860972 4767 webserver.cc:460] Webserver started at http://127.12.234.62:38645/ using document root <none> and password file <none>
I20260809 04:49:15.861596 4767 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:15.861696 4767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:15.861955 4767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:15.864785 4767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "bce5cb86bfc44c45b4a8d7a0a3a790d5"
format_stamp: "Formatted at 2026-08-09 04:49:15 on dist-test-slave-1jjb"
server_key: "1f251e1c951893982c872e40fec21f49"
server_key_iv: "1161088586e0d3c9f3b66c58577a5ad3"
server_key_version: "encryptionkey@0"
I20260809 04:49:15.865453 4767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "bce5cb86bfc44c45b4a8d7a0a3a790d5"
format_stamp: "Formatted at 2026-08-09 04:49:15 on dist-test-slave-1jjb"
server_key: "1f251e1c951893982c872e40fec21f49"
server_key_iv: "1161088586e0d3c9f3b66c58577a5ad3"
server_key_version: "encryptionkey@0"
I20260809 04:49:15.870102 4767 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 04:49:15.873368 4783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:15.874606 4767 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 04:49:15.874771 4767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "bce5cb86bfc44c45b4a8d7a0a3a790d5"
format_stamp: "Formatted at 2026-08-09 04:49:15 on dist-test-slave-1jjb"
server_key: "1f251e1c951893982c872e40fec21f49"
server_key_iv: "1161088586e0d3c9f3b66c58577a5ad3"
server_key_version: "encryptionkey@0"
I20260809 04:49:15.874941 4767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:15.897437 4767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:15.900910 4767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:15.901171 4767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:15.918308 4767 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:46025
I20260809 04:49:15.918346 4838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:46025 every 8 connection(s)
I20260809 04:49:15.919485 4767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:15.922809 4839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:15.925047 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 4767
I20260809 04:49:15.925206 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:49:15.925693 13224 external_mini_cluster.cc:1428] Setting key 350f3436bf32b9b206ad046ad4e83563
I20260809 04:49:15.934118 4839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bce5cb86bfc44c45b4a8d7a0a3a790d5: Bootstrap starting.
I20260809 04:49:15.937462 4839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bce5cb86bfc44c45b4a8d7a0a3a790d5: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:15.938555 4839 log.cc:824] T 00000000000000000000000000000000 P bce5cb86bfc44c45b4a8d7a0a3a790d5: Log is configured to *not* fsync() on all Append() calls
Aug 09 04:49:15 dist-test-slave-1jjb krb5kdc[4754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:49:15.940958 4839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bce5cb86bfc44c45b4a8d7a0a3a790d5: No bootstrap required, opened a new log
I20260809 04:49:15.946127 4842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:15.927329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39193 (local address 127.12.234.62:46025)
0809 04:49:15.928135 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:15.928148 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:15.928195 (+ 47us) server_negotiation.cc:407] Connection header received
0809 04:49:15.929673 (+ 1478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:15.929692 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:15.930011 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:15.930285 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:15.931871 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:15.933156 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:15.934749 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:15.935292 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:15.935428 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:15.939666 (+ 4238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:15.939673 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:15.939684 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:15.939718 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:15.942551 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:15.943166 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:15.943172 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:15.943177 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:15.943328 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:15.943677 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:15.943681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:15.943685 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:15.944175 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:15.944448 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:15.944902 (+ 454us) server_negotiation.cc:299] Negotiation successful
0809 04:49:15.945259 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":318,"threads_started":1}
I20260809 04:49:15.947929 4839 raft_consensus.cc:348] T 00000000000000000000000000000000 P bce5cb86bfc44c45b4a8d7a0a3a790d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bce5cb86bfc44c45b4a8d7a0a3a790d5" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46025 } }
I20260809 04:49:15.948354 4839 raft_consensus.cc:374] T 00000000000000000000000000000000 P bce5cb86bfc44c45b4a8d7a0a3a790d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:15.948460 4839 raft_consensus.cc:729] T 00000000000000000000000000000000 P bce5cb86bfc44c45b4a8d7a0a3a790d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bce5cb86bfc44c45b4a8d7a0a3a790d5, State: Initialized, Role: FOLLOWER
I20260809 04:49:15.948984 4839 consensus_queue.cc:260] T 00000000000000000000000000000000 P bce5cb86bfc44c45b4a8d7a0a3a790d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bce5cb86bfc44c45b4a8d7a0a3a790d5" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46025 } }
I20260809 04:49:15.949270 4839 raft_consensus.cc:388] T 00000000000000000000000000000000 P bce5cb86bfc44c45b4a8d7a0a3a790d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:15.949335 13224 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:15.949426 4839 raft_consensus.cc:482] T 00000000000000000000000000000000 P bce5cb86bfc44c45b4a8d7a0a3a790d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:15.949565 4839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bce5cb86bfc44c45b4a8d7a0a3a790d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:15.952453 4839 raft_consensus.cc:504] T 00000000000000000000000000000000 P bce5cb86bfc44c45b4a8d7a0a3a790d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bce5cb86bfc44c45b4a8d7a0a3a790d5" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46025 } }
I20260809 04:49:15.953044 4839 leader_election.cc:302] T 00000000000000000000000000000000 P bce5cb86bfc44c45b4a8d7a0a3a790d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bce5cb86bfc44c45b4a8d7a0a3a790d5; no voters:
I20260809 04:49:15.953624 4839 leader_election.cc:288] T 00000000000000000000000000000000 P bce5cb86bfc44c45b4a8d7a0a3a790d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:15.953688 4848 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bce5cb86bfc44c45b4a8d7a0a3a790d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:15.955039 4848 raft_consensus.cc:686] T 00000000000000000000000000000000 P bce5cb86bfc44c45b4a8d7a0a3a790d5 [term 1 LEADER]: Becoming Leader. State: Replica: bce5cb86bfc44c45b4a8d7a0a3a790d5, State: Running, Role: LEADER
I20260809 04:49:15.955587 4839 sys_catalog.cc:564] T 00000000000000000000000000000000 P bce5cb86bfc44c45b4a8d7a0a3a790d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:15.955523 4848 consensus_queue.cc:237] T 00000000000000000000000000000000 P bce5cb86bfc44c45b4a8d7a0a3a790d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bce5cb86bfc44c45b4a8d7a0a3a790d5" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46025 } }
I20260809 04:49:15.964020 4851 sys_catalog.cc:455] T 00000000000000000000000000000000 P bce5cb86bfc44c45b4a8d7a0a3a790d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bce5cb86bfc44c45b4a8d7a0a3a790d5. Latest consensus state: current_term: 1 leader_uuid: "bce5cb86bfc44c45b4a8d7a0a3a790d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bce5cb86bfc44c45b4a8d7a0a3a790d5" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46025 } } }
I20260809 04:49:15.964447 4851 sys_catalog.cc:458] T 00000000000000000000000000000000 P bce5cb86bfc44c45b4a8d7a0a3a790d5 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:15.965823 4849 sys_catalog.cc:455] T 00000000000000000000000000000000 P bce5cb86bfc44c45b4a8d7a0a3a790d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bce5cb86bfc44c45b4a8d7a0a3a790d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bce5cb86bfc44c45b4a8d7a0a3a790d5" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46025 } } }
I20260809 04:49:15.966135 4849 sys_catalog.cc:458] T 00000000000000000000000000000000 P bce5cb86bfc44c45b4a8d7a0a3a790d5 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:15.967135 4857 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:15.971565 4842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:15.954444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46497 (local address 127.12.234.62:46025)
0809 04:49:15.954575 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:15.954580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:15.958162 (+ 3582us) server_negotiation.cc:407] Connection header received
0809 04:49:15.958329 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:15.958332 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:15.958380 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:15.958486 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:15.959724 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:15.960487 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:15.961651 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:15.962215 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:15.962347 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:15.965407 (+ 3060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:15.965416 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:15.965422 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:15.965464 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:15.968619 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:15.969419 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:15.969424 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:15.969429 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:15.969612 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:15.970169 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:15.970174 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:15.970180 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:15.970490 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:15.970730 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:15.971025 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 04:49:15.971241 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:49:15.977859 4857 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:15.985745 4857 catalog_manager.cc:1347] Generated new cluster ID: cfa5b981167148268cb6ae4f03f441f4
I20260809 04:49:15.985839 4857 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:16.009258 4857 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:16.011360 4857 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:16.017678 4857 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bce5cb86bfc44c45b4a8d7a0a3a790d5: Generated new TSK 0
I20260809 04:49:16.018535 4857 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:16.035876 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 4767
I20260809 04:49:16.037351 4833 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:49:16.121284 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 4767
2026-08-09T04:49:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1997 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:16 dist-test-slave-1jjb krb5kdc[4874](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:16 dist-test-slave-1jjb krb5kdc[4874](info): set up 2 sockets
Aug 09 04:49:16 dist-test-slave-1jjb krb5kdc[4874](info): commencing operation
krb5kdc: starting...
W20260809 04:49:17.225667 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:49:17 dist-test-slave-1jjb krb5kdc[4874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:49:17.496490 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:43337
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:33027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:43337
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:17.804963 4887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:17.805203 4887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:17.805285 4887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:17.812156 4887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:17.812253 4887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:17.812305 4887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:17.812372 4887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:17.812418 4887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:17.821359 4887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:33027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:43337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:43337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:49:17.822563 4887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:17.823745 4887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:17.834717 4892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:17.834769 4893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:17.834831 4896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:17.835721 4887 server_base.cc:1034] running on GCE node
I20260809 04:49:17.836450 4887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:17.837669 4887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:17.838908 4887 hybrid_clock.cc:648] HybridClock initialized: now 1786250957838867 us; error 57 us; skew 500 ppm
I20260809 04:49:17.839205 4887 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:49:17 dist-test-slave-1jjb krb5kdc[4874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250957, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:17.842258 4887 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:49:17.843643 4887 webserver.cc:460] Webserver started at http://127.12.234.62:40783/ using document root <none> and password file <none>
I20260809 04:49:17.844311 4887 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:17.844457 4887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:17.844734 4887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:17.847316 4887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "460c3e9ce8a842e8b965597d499f070b"
format_stamp: "Formatted at 2026-08-09 04:49:17 on dist-test-slave-1jjb"
server_key: "a25c1ef5a4e2ebaf9eec67ab90c7b21d"
server_key_iv: "b8fdd54d5d3284814f2af0ef57b0b0b0"
server_key_version: "encryptionkey@0"
I20260809 04:49:17.848042 4887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "460c3e9ce8a842e8b965597d499f070b"
format_stamp: "Formatted at 2026-08-09 04:49:17 on dist-test-slave-1jjb"
server_key: "a25c1ef5a4e2ebaf9eec67ab90c7b21d"
server_key_iv: "b8fdd54d5d3284814f2af0ef57b0b0b0"
server_key_version: "encryptionkey@0"
I20260809 04:49:17.853031 4887 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260809 04:49:17.856294 4903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:17.857450 4887 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 04:49:17.857623 4887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "460c3e9ce8a842e8b965597d499f070b"
format_stamp: "Formatted at 2026-08-09 04:49:17 on dist-test-slave-1jjb"
server_key: "a25c1ef5a4e2ebaf9eec67ab90c7b21d"
server_key_iv: "b8fdd54d5d3284814f2af0ef57b0b0b0"
server_key_version: "encryptionkey@0"
I20260809 04:49:17.857821 4887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:17.874032 4887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:17.877653 4887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:17.877915 4887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:17.895598 4887 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:43337
I20260809 04:49:17.895620 4957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:43337 every 8 connection(s)
I20260809 04:49:17.896781 4887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:17.898890 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 4887
I20260809 04:49:17.899047 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:49:17.899447 13224 external_mini_cluster.cc:1428] Setting key 887634df8ec8c185b4c64d81baed9837
I20260809 04:49:17.900473 4958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:17.913031 4961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:17.901114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53145 (local address 127.12.234.62:43337)
0809 04:49:17.901854 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:17.901869 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:17.901916 (+ 47us) server_negotiation.cc:407] Connection header received
0809 04:49:17.903873 (+ 1957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:17.903900 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:17.904403 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:17.904779 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:17.905208 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:17.907058 (+ 1850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:17.908245 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:17.909209 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:17.909410 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:17.910460 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:17.910469 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:17.910485 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:17.911130 (+ 645us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:17.911231 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:17.911258 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:17.911371 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:17.911523 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 04:49:17.911788 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":307,"threads_started":1}
I20260809 04:49:17.914338 4958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 460c3e9ce8a842e8b965597d499f070b: Bootstrap starting.
I20260809 04:49:17.916155 13224 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:17.917196 4958 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 460c3e9ce8a842e8b965597d499f070b: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:17.918187 4958 log.cc:824] T 00000000000000000000000000000000 P 460c3e9ce8a842e8b965597d499f070b: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:17.920949 4958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 460c3e9ce8a842e8b965597d499f070b: No bootstrap required, opened a new log
I20260809 04:49:17.926185 4961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:17.920243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54511 (local address 127.12.234.62:43337)
0809 04:49:17.920418 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:17.920423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:17.920916 (+ 493us) server_negotiation.cc:407] Connection header received
0809 04:49:17.921017 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:17.921021 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:17.921078 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:17.921168 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:17.921591 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:17.922652 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:17.923918 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:17.924594 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:17.924711 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:17.925520 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:17.925525 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:17.925529 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:17.925547 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:17.925591 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:17.925612 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:17.925695 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:17.925830 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 04:49:17.925900 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 04:49:17.928869 4958 raft_consensus.cc:348] T 00000000000000000000000000000000 P 460c3e9ce8a842e8b965597d499f070b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460c3e9ce8a842e8b965597d499f070b" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 43337 } }
I20260809 04:49:17.929217 4958 raft_consensus.cc:374] T 00000000000000000000000000000000 P 460c3e9ce8a842e8b965597d499f070b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:17.929324 4958 raft_consensus.cc:729] T 00000000000000000000000000000000 P 460c3e9ce8a842e8b965597d499f070b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 460c3e9ce8a842e8b965597d499f070b, State: Initialized, Role: FOLLOWER
I20260809 04:49:17.929857 4958 consensus_queue.cc:260] T 00000000000000000000000000000000 P 460c3e9ce8a842e8b965597d499f070b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460c3e9ce8a842e8b965597d499f070b" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 43337 } }
I20260809 04:49:17.930084 4958 raft_consensus.cc:388] T 00000000000000000000000000000000 P 460c3e9ce8a842e8b965597d499f070b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:17.930236 4958 raft_consensus.cc:482] T 00000000000000000000000000000000 P 460c3e9ce8a842e8b965597d499f070b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:17.930390 4958 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 460c3e9ce8a842e8b965597d499f070b [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:17.932585 4958 raft_consensus.cc:504] T 00000000000000000000000000000000 P 460c3e9ce8a842e8b965597d499f070b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460c3e9ce8a842e8b965597d499f070b" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 43337 } }
I20260809 04:49:17.933051 4958 leader_election.cc:302] T 00000000000000000000000000000000 P 460c3e9ce8a842e8b965597d499f070b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 460c3e9ce8a842e8b965597d499f070b; no voters:
I20260809 04:49:17.933539 4958 leader_election.cc:288] T 00000000000000000000000000000000 P 460c3e9ce8a842e8b965597d499f070b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:17.933660 4969 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 460c3e9ce8a842e8b965597d499f070b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:17.934371 4969 raft_consensus.cc:686] T 00000000000000000000000000000000 P 460c3e9ce8a842e8b965597d499f070b [term 1 LEADER]: Becoming Leader. State: Replica: 460c3e9ce8a842e8b965597d499f070b, State: Running, Role: LEADER
I20260809 04:49:17.934839 4969 consensus_queue.cc:237] T 00000000000000000000000000000000 P 460c3e9ce8a842e8b965597d499f070b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460c3e9ce8a842e8b965597d499f070b" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 43337 } }
I20260809 04:49:17.935442 4958 sys_catalog.cc:564] T 00000000000000000000000000000000 P 460c3e9ce8a842e8b965597d499f070b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:17.937711 4972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460c3e9ce8a842e8b965597d499f070b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 460c3e9ce8a842e8b965597d499f070b. Latest consensus state: current_term: 1 leader_uuid: "460c3e9ce8a842e8b965597d499f070b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460c3e9ce8a842e8b965597d499f070b" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 43337 } } }
I20260809 04:49:17.938066 4972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460c3e9ce8a842e8b965597d499f070b [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:17.939594 4970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460c3e9ce8a842e8b965597d499f070b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "460c3e9ce8a842e8b965597d499f070b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "460c3e9ce8a842e8b965597d499f070b" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 43337 } } }
I20260809 04:49:17.939883 4970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460c3e9ce8a842e8b965597d499f070b [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:17.940339 4979 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:17.944305 4979 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:17.951803 4979 catalog_manager.cc:1347] Generated new cluster ID: 629ffebcb68b487e946ddb97c1c9c49f
I20260809 04:49:17.951908 4979 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:17.975054 4979 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:17.976156 4979 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:17.986271 4979 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 460c3e9ce8a842e8b965597d499f070b: Generated new TSK 0
I20260809 04:49:17.986863 4979 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:17.992933 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 4887
W20260809 04:49:17.994092 4953 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53145
W20260809 04:49:17.994691 13224 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 4887: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:49:17.994823 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 4887
2026-08-09T04:49:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1873 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:18 dist-test-slave-1jjb krb5kdc[4991](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:18 dist-test-slave-1jjb krb5kdc[4991](info): set up 2 sockets
Aug 09 04:49:18 dist-test-slave-1jjb krb5kdc[4991](info): commencing operation
krb5kdc: starting...
W20260809 04:49:19.076571 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.005s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:49:19 dist-test-slave-1jjb krb5kdc[4991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:49:19.345259 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:46577
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:42801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:46577
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:19.664237 5006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:19.664510 5006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:19.664672 5006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:19.671588 5006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:19.671689 5006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:19.671733 5006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:19.671844 5006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:19.671955 5006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:19.681257 5006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:42801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:46577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:46577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:49:19.682371 5006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:19.683598 5006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:19.694888 5012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:19.694942 5011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:19.694962 5014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:19.695873 5006 server_base.cc:1034] running on GCE node
I20260809 04:49:19.696584 5006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:19.697798 5006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:19.698997 5006 hybrid_clock.cc:648] HybridClock initialized: now 1786250959698963 us; error 27 us; skew 500 ppm
I20260809 04:49:19.699297 5006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:49:19 dist-test-slave-1jjb krb5kdc[4991](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250959, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:19.708534 5006 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:49:19.709836 5006 webserver.cc:460] Webserver started at http://127.12.234.62:44511/ using document root <none> and password file <none>
I20260809 04:49:19.710455 5006 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:19.710588 5006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:19.710855 5006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:19.713627 5006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "348ba4df915b469880d8bdb57400cbae"
format_stamp: "Formatted at 2026-08-09 04:49:19 on dist-test-slave-1jjb"
server_key: "2a3f4d6c8cb23e6a5f888f1dc733fba7"
server_key_iv: "8b3c887cdcaf7fa53dc53bbdb2eb4f49"
server_key_version: "encryptionkey@0"
I20260809 04:49:19.714423 5006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "348ba4df915b469880d8bdb57400cbae"
format_stamp: "Formatted at 2026-08-09 04:49:19 on dist-test-slave-1jjb"
server_key: "2a3f4d6c8cb23e6a5f888f1dc733fba7"
server_key_iv: "8b3c887cdcaf7fa53dc53bbdb2eb4f49"
server_key_version: "encryptionkey@0"
I20260809 04:49:19.719116 5006 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 04:49:19.722325 5022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:19.723907 5006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 04:49:19.724128 5006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "348ba4df915b469880d8bdb57400cbae"
format_stamp: "Formatted at 2026-08-09 04:49:19 on dist-test-slave-1jjb"
server_key: "2a3f4d6c8cb23e6a5f888f1dc733fba7"
server_key_iv: "8b3c887cdcaf7fa53dc53bbdb2eb4f49"
server_key_version: "encryptionkey@0"
I20260809 04:49:19.724303 5006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:19.729290 5006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:19.732816 5006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:19.733119 5006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:19.750407 5006 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:46577
I20260809 04:49:19.750514 5077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:46577 every 8 connection(s)
I20260809 04:49:19.751612 5006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:19.755167 5078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:19.759493 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 5006
I20260809 04:49:19.759675 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:49:19.760279 13224 external_mini_cluster.cc:1428] Setting key 00156746a698144075a2a537ed19d18d
I20260809 04:49:19.766599 5078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 348ba4df915b469880d8bdb57400cbae: Bootstrap starting.
I20260809 04:49:19.767872 5081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:19.761868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59593 (local address 127.12.234.62:46577)
0809 04:49:19.762717 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:19.762727 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:19.762763 (+ 36us) server_negotiation.cc:407] Connection header received
0809 04:49:19.764857 (+ 2094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:19.764876 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:19.765139 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:19.765470 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:19.766555 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:19.766560 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:19.766570 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:19.766583 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:19.766634 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:19.766650 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:19.766859 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:19.766975 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 04:49:19.767126 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":395,"threads_started":1}
I20260809 04:49:19.770249 5078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 348ba4df915b469880d8bdb57400cbae: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:19.771375 13224 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:19.771569 5078 log.cc:824] T 00000000000000000000000000000000 P 348ba4df915b469880d8bdb57400cbae: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:19.773991 5078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 348ba4df915b469880d8bdb57400cbae: No bootstrap required, opened a new log
I20260809 04:49:19.777797 5081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:19.775563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51697 (local address 127.12.234.62:46577)
0809 04:49:19.775764 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:19.775770 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:19.776165 (+ 395us) server_negotiation.cc:407] Connection header received
0809 04:49:19.776296 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:19.776301 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:19.776348 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:19.776450 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:19.777211 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:19.777217 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:19.777221 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 04:49:19.777237 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:19.777277 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 04:49:19.777299 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:19.777397 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:19.777482 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 04:49:19.777528 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 04:49:19.781536 5078 raft_consensus.cc:348] T 00000000000000000000000000000000 P 348ba4df915b469880d8bdb57400cbae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "348ba4df915b469880d8bdb57400cbae" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46577 } }
I20260809 04:49:19.781836 5078 raft_consensus.cc:374] T 00000000000000000000000000000000 P 348ba4df915b469880d8bdb57400cbae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 04:49:19.781973 5078 raft_consensus.cc:729] T 00000000000000000000000000000000 P 348ba4df915b469880d8bdb57400cbae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 348ba4df915b469880d8bdb57400cbae, State: Initialized, Role: FOLLOWER
I20260809 04:49:19.782550 5078 consensus_queue.cc:260] T 00000000000000000000000000000000 P 348ba4df915b469880d8bdb57400cbae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "348ba4df915b469880d8bdb57400cbae" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46577 } }
I20260809 04:49:19.782790 5078 raft_consensus.cc:388] T 00000000000000000000000000000000 P 348ba4df915b469880d8bdb57400cbae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 04:49:19.782943 5078 raft_consensus.cc:482] T 00000000000000000000000000000000 P 348ba4df915b469880d8bdb57400cbae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 04:49:19.783141 5078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 348ba4df915b469880d8bdb57400cbae [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:19.785320 5078 raft_consensus.cc:504] T 00000000000000000000000000000000 P 348ba4df915b469880d8bdb57400cbae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "348ba4df915b469880d8bdb57400cbae" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46577 } }
I20260809 04:49:19.785830 5078 leader_election.cc:302] T 00000000000000000000000000000000 P 348ba4df915b469880d8bdb57400cbae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 348ba4df915b469880d8bdb57400cbae; no voters:
I20260809 04:49:19.786368 5078 leader_election.cc:288] T 00000000000000000000000000000000 P 348ba4df915b469880d8bdb57400cbae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 04:49:19.786451 5088 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 348ba4df915b469880d8bdb57400cbae [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:19.787086 5088 raft_consensus.cc:686] T 00000000000000000000000000000000 P 348ba4df915b469880d8bdb57400cbae [term 1 LEADER]: Becoming Leader. State: Replica: 348ba4df915b469880d8bdb57400cbae, State: Running, Role: LEADER
I20260809 04:49:19.787567 5088 consensus_queue.cc:237] T 00000000000000000000000000000000 P 348ba4df915b469880d8bdb57400cbae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "348ba4df915b469880d8bdb57400cbae" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46577 } }
I20260809 04:49:19.788172 5078 sys_catalog.cc:564] T 00000000000000000000000000000000 P 348ba4df915b469880d8bdb57400cbae [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:19.790241 5090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 348ba4df915b469880d8bdb57400cbae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 348ba4df915b469880d8bdb57400cbae. Latest consensus state: current_term: 1 leader_uuid: "348ba4df915b469880d8bdb57400cbae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "348ba4df915b469880d8bdb57400cbae" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46577 } } }
I20260809 04:49:19.790700 5090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 348ba4df915b469880d8bdb57400cbae [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:19.791909 5096 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:19.791842 5089 sys_catalog.cc:455] T 00000000000000000000000000000000 P 348ba4df915b469880d8bdb57400cbae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "348ba4df915b469880d8bdb57400cbae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "348ba4df915b469880d8bdb57400cbae" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46577 } } }
I20260809 04:49:19.792516 5089 sys_catalog.cc:458] T 00000000000000000000000000000000 P 348ba4df915b469880d8bdb57400cbae [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:19.798123 5096 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:19.805868 5096 catalog_manager.cc:1347] Generated new cluster ID: 8b198852122f4f5082e061794f20f6a5
I20260809 04:49:19.805964 5096 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:19.832644 5096 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:19.833031 5096 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:19.841629 5096 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 348ba4df915b469880d8bdb57400cbae: Generated new TSK 0
I20260809 04:49:19.842295 5096 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:19.854126 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 5006
W20260809 04:49:19.855556 5073 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59593
W20260809 04:49:19.856166 13224 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 5006: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 04:49:19.856295 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 5006
2026-08-09T04:49:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1861 ms)
[----------] 18 tests from AuthTokenIssuingTest (33992 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:19 dist-test-slave-1jjb krb5kdc[5109](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:19 dist-test-slave-1jjb krb5kdc[5109](info): set up 2 sockets
Aug 09 04:49:19 dist-test-slave-1jjb krb5kdc[5109](info): commencing operation
krb5kdc: starting...
W20260809 04:49:20.956243 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.003s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:49:21 dist-test-slave-1jjb krb5kdc[5109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:49:21.235191 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:39467
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:38245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:39467,127.12.234.61:33601,127.12.234.60:39615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:21.561437 5122 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 04:49:21.561692 5122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:21.561749 5122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:21.561820 5122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:21.568189 5122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:21.568254 5122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:21.568310 5122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:21.568356 5122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:21.568408 5122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:21.576872 5122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:38245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:39467,127.12.234.61:33601,127.12.234.60:39615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:39467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:49:21.577814 5122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:21.579026 5122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:21.590045 5127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:21.590097 5130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:21.590132 5128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:21.591063 5122 server_base.cc:1034] running on GCE node
I20260809 04:49:21.591751 5122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:21.593086 5122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:21.594416 5122 hybrid_clock.cc:648] HybridClock initialized: now 1786250961594343 us; error 53 us; skew 500 ppm
I20260809 04:49:21.594779 5122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:49:21 dist-test-slave-1jjb krb5kdc[5109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250961, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:21.598001 5122 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:49:21.599331 5122 webserver.cc:460] Webserver started at http://127.12.234.62:40007/ using document root <none> and password file <none>
I20260809 04:49:21.599900 5122 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:21.599985 5122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:21.600395 5122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:21.603379 5122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "938a8c77fa414a5ab9462d7f442eec2c"
format_stamp: "Formatted at 2026-08-09 04:49:21 on dist-test-slave-1jjb"
server_key: "e0f201c3d91bbad4076c6105f1f61adc"
server_key_iv: "a592bcec1ab264d250a7687830b6b1d6"
server_key_version: "encryptionkey@0"
I20260809 04:49:21.604146 5122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "938a8c77fa414a5ab9462d7f442eec2c"
format_stamp: "Formatted at 2026-08-09 04:49:21 on dist-test-slave-1jjb"
server_key: "e0f201c3d91bbad4076c6105f1f61adc"
server_key_iv: "a592bcec1ab264d250a7687830b6b1d6"
server_key_version: "encryptionkey@0"
I20260809 04:49:21.609227 5122 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 04:49:21.612813 5138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:21.613988 5122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:49:21.614176 5122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "938a8c77fa414a5ab9462d7f442eec2c"
format_stamp: "Formatted at 2026-08-09 04:49:21 on dist-test-slave-1jjb"
server_key: "e0f201c3d91bbad4076c6105f1f61adc"
server_key_iv: "a592bcec1ab264d250a7687830b6b1d6"
server_key_version: "encryptionkey@0"
I20260809 04:49:21.614399 5122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:21.634518 5122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:21.638731 5122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:21.639005 5122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:21.656291 5122 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:39467
I20260809 04:49:21.656314 5192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:39467 every 8 connection(s)
I20260809 04:49:21.657465 5122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:21.660749 5194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:21.661868 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 5122
I20260809 04:49:21.662076 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:49:21.662748 13224 external_mini_cluster.cc:1428] Setting key cad82be9f33190fe2d464b2fdbdc30f6
I20260809 04:49:21.666859 5194 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.62" port: 39467 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:49:21 dist-test-slave-1jjb krb5kdc[5109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250961, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:49:21.686908 5196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:21.669603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45266 (local address 127.12.234.62:39467)
0809 04:49:21.670385 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:21.670399 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:21.670798 (+ 399us) server_negotiation.cc:407] Connection header received
0809 04:49:21.672651 (+ 1853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:21.672657 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:21.672864 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:21.673066 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:21.674245 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:21.675312 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:21.676600 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:21.677096 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:21.677264 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:21.680425 (+ 3161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:21.680432 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:21.680442 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:21.680474 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:21.683112 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:21.683737 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:21.683743 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:21.683745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:21.683871 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:21.684310 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:21.684318 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:21.684326 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:21.684710 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:21.684902 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:21.685394 (+ 492us) server_negotiation.cc:299] Negotiation successful
0809 04:49:21.685666 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":342,"threads_started":1}
I20260809 04:49:21.686889 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:21.669616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:39467 (local address 127.0.0.1:45266)
0809 04:49:21.670649 (+ 1033us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:21.670679 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:21.672305 (+ 1626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:21.673079 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:21.673086 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:21.673766 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:21.674041 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:21.674055 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:21.675452 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:21.675459 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:21.676472 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:21.676484 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:21.677295 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:21.677300 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:21.677464 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:21.678036 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:21.678054 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:21.680241 (+ 2187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:21.683249 (+ 3008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:21.683260 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:21.683269 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:21.683611 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:21.684002 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:21.684006 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:21.684010 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:21.684205 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:21.684852 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:21.684858 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:21.685053 (+ 195us) client_negotiation.cc:769] Sending connection context
0809 04:49:21.685267 (+ 214us) client_negotiation.cc:240] Negotiation successful
0809 04:49:21.685794 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":874,"mutex_wait_us":159,"thread_start_us":686,"threads_started":1}
I20260809 04:49:21.689471 5194 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.61" port: 33601 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:49:21.690508 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:21.690129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:33601 (local address 127.0.0.1:38402)
0809 04:49:21.690290 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:21.690399 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 04:49:21.690788 5139 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.234.61:33601: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.12.234.61@KRBTEST.COM; defaulting to no policy
I20260809 04:49:21.692274 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:21.691846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:33601 (local address 127.0.0.1:38408)
0809 04:49:21.691967 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:21.692025 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 04:49:21.692559 5194 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.61:33601: Network error: Client connection negotiation failed: client connection to 127.12.234.61:33601: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.61.keytab.
Entry for principal kudu/127.12.234.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.61.keytab.
I20260809 04:49:21.724540 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.61:33601
--webserver_interface=127.12.234.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.61.keytab
--principal=kudu/127.12.234.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:38245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:39467,127.12.234.61:33601,127.12.234.60:39615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:49:21.741828 5194 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.61" port: 33601 } attempt: 1
I20260809 04:49:21.742955 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:21.742533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:33601 (local address 127.0.0.1:38412)
0809 04:49:21.742715 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:21.742814 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 04:49:21.743937 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:21.743629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:33601 (local address 127.0.0.1:38420)
0809 04:49:21.743772 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:21.743829 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 04:49:21.744114 5194 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.61:33601: Network error: Client connection negotiation failed: client connection to 127.12.234.61:33601: connect: Connection refused (error 111)
I20260809 04:49:21.812347 5194 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.61" port: 33601 } attempt: 2
I20260809 04:49:21.813513 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:21.813031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:33601 (local address 127.0.0.1:38426)
0809 04:49:21.813283 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:21.813377 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 04:49:21.814617 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:21.814219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:33601 (local address 127.0.0.1:38432)
0809 04:49:21.814371 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:21.814473 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 04:49:21.814812 5194 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.61:33601: Network error: Client connection negotiation failed: client connection to 127.12.234.61:33601: connect: Connection refused (error 111)
I20260809 04:49:21.906067 5194 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.61" port: 33601 } attempt: 3
I20260809 04:49:21.907169 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:21.906730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:33601 (local address 127.0.0.1:38446)
0809 04:49:21.906947 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:21.907038 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 04:49:21.908321 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:21.907938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:33601 (local address 127.0.0.1:38458)
0809 04:49:21.908101 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:21.908176 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 04:49:21.908519 5194 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.61:33601: Network error: Client connection negotiation failed: client connection to 127.12.234.61:33601: connect: Connection refused (error 111)
W20260809 04:49:22.035144 5202 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 04:49:22.035472 5202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:22.035588 5202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:22.035709 5202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:22.042177 5202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:22.042284 5202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:22.042389 5202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:22.042475 5202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:22.042564 5202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:22.051190 5202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:38245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:39467,127.12.234.61:33601,127.12.234.60:39615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.61.keytab
--principal=kudu/127.12.234.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.61:33601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.234.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:49:22.051726 5194 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.61" port: 33601 } attempt: 4
I20260809 04:49:22.052412 5202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:22.052933 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.052382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:33601 (local address 127.0.0.1:38460)
0809 04:49:22.052678 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.052763 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260809 04:49:22.053791 5202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 04:49:22.053934 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.053614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:33601 (local address 127.0.0.1:38476)
0809 04:49:22.053747 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.053811 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 04:49:22.054095 5194 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.61:33601: Network error: Client connection negotiation failed: client connection to 127.12.234.61:33601: connect: Connection refused (error 111)
W20260809 04:49:22.064419 5207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:22.064383 5210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:22.064543 5208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:22.065840 5202 server_base.cc:1034] running on GCE node
I20260809 04:49:22.066522 5202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:22.067716 5202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:22.068933 5202 hybrid_clock.cc:648] HybridClock initialized: now 1786250962068852 us; error 46 us; skew 500 ppm
I20260809 04:49:22.069319 5202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:49:22 dist-test-slave-1jjb krb5kdc[5109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250962, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:22.072589 5202 init.cc:377] Logged in from keytab as kudu/127.12.234.61@KRBTEST.COM (short username kudu)
I20260809 04:49:22.073973 5202 webserver.cc:460] Webserver started at http://127.12.234.61:39427/ using document root <none> and password file <none>
I20260809 04:49:22.074666 5202 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:22.074764 5202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:22.075043 5202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:22.077756 5202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-1/data/instance:
uuid: "d953ba5fa0a749eeb1c6c1fa58a7a0e4"
format_stamp: "Formatted at 2026-08-09 04:49:22 on dist-test-slave-1jjb"
server_key: "6b4795a22ca5fb5e2170959ebaf2aa00"
server_key_iv: "b4a184946024e7239545cd399d3d7b12"
server_key_version: "encryptionkey@0"
I20260809 04:49:22.078478 5202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-1/wal/instance:
uuid: "d953ba5fa0a749eeb1c6c1fa58a7a0e4"
format_stamp: "Formatted at 2026-08-09 04:49:22 on dist-test-slave-1jjb"
server_key: "6b4795a22ca5fb5e2170959ebaf2aa00"
server_key_iv: "b4a184946024e7239545cd399d3d7b12"
server_key_version: "encryptionkey@0"
I20260809 04:49:22.083599 5202 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 04:49:22.086871 5218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:22.088128 5202 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 04:49:22.088317 5202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-1/wal
uuid: "d953ba5fa0a749eeb1c6c1fa58a7a0e4"
format_stamp: "Formatted at 2026-08-09 04:49:22 on dist-test-slave-1jjb"
server_key: "6b4795a22ca5fb5e2170959ebaf2aa00"
server_key_iv: "b4a184946024e7239545cd399d3d7b12"
server_key_version: "encryptionkey@0"
I20260809 04:49:22.088596 5202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:22.112865 5202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:22.116629 5202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:22.116897 5202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:22.133809 5202 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.61:33601
I20260809 04:49:22.133829 5269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.61:33601 every 8 connection(s)
I20260809 04:49:22.135022 5202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-1/data/info.pb
I20260809 04:49:22.138397 5270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:22.144446 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 5202
I20260809 04:49:22.144619 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-1/wal/instance
I20260809 04:49:22.145119 13224 external_mini_cluster.cc:1428] Setting key 416dbf88068fd1740b5abfb490d8802a
I20260809 04:49:22.144716 5270 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.62" port: 39467 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:49:22 dist-test-slave-1jjb krb5kdc[5109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250962, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.61@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:49:22.168007 5196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.147476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45282 (local address 127.12.234.62:39467)
0809 04:49:22.147670 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:22.147675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:22.148864 (+ 1189us) server_negotiation.cc:407] Connection header received
0809 04:49:22.150530 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:22.150538 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:22.150624 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:22.150755 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:22.152288 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.153404 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.155653 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.156426 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.156542 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:22.160933 (+ 4391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:22.160942 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:22.160948 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:22.160993 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:22.164849 (+ 3856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.165602 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.165610 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.165615 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.165758 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.166168 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.166175 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.166180 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.166532 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:22.166713 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:22.167374 (+ 661us) server_negotiation.cc:299] Negotiation successful
0809 04:49:22.167616 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 04:49:22.168567 5272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.147924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:39467 (local address 127.0.0.1:45282)
0809 04:49:22.148686 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.148733 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:22.150289 (+ 1556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:22.150939 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:22.150946 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:22.151384 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:22.151991 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:22.152014 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.153594 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.153601 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:22.155457 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:22.155475 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.156906 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.156914 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:22.157243 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:22.157961 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:22.157992 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:22.160694 (+ 2702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:22.165032 (+ 4338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:22.165041 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:22.165050 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:22.165463 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:22.165864 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:22.165867 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:22.165870 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:22.166040 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:22.166726 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:22.166732 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:22.167009 (+ 277us) client_negotiation.cc:769] Sending connection context
0809 04:49:22.167208 (+ 199us) client_negotiation.cc:240] Negotiation successful
0809 04:49:22.167556 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":375,"threads_started":1}
I20260809 04:49:22.170151 5270 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.61" port: 33601 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:49:22 dist-test-slave-1jjb krb5kdc[5109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250962, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.61@KRBTEST.COM for kudu/127.12.234.61@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.234.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.60@KRBTEST.COM" created.
I20260809 04:49:22.183458 5272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.170951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:33601 (local address 127.0.0.1:38478)
0809 04:49:22.171185 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.171207 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:22.171320 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:22.172234 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:22.172238 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:22.172641 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:22.172760 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:22.172768 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.174020 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.174027 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:22.174902 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:22.174913 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.175597 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.175602 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:22.175770 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:22.176318 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:22.176333 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:22.178046 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:22.181236 (+ 3190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:22.181242 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:22.181245 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:22.181532 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:22.181864 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:22.181868 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:22.181871 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:22.181981 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:22.182536 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:22.182541 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:22.182675 (+ 134us) client_negotiation.cc:769] Sending connection context
0809 04:49:22.182822 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 04:49:22.183077 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 04:49:22.183565 5274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.170952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38478 (local address 127.12.234.61:33601)
0809 04:49:22.171830 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:22.171835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:22.171862 (+ 27us) server_negotiation.cc:407] Connection header received
0809 04:49:22.171896 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:22.171900 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:22.172028 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:22.172230 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:22.172939 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.173900 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.175014 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.175471 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.175590 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:22.178238 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:22.178248 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:22.178258 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:22.178288 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:22.181133 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.181642 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.181648 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.181651 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.181757 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.182083 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.182087 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.182090 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.182413 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:22.182622 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:22.182975 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 04:49:22.183176 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":747,"thread_start_us":618,"threads_started":1}
I20260809 04:49:22.185604 5270 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:49:22.186684 5272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.186095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:39615 (local address 127.0.0.1:37422)
0809 04:49:22.186289 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.186499 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260809 04:49:22.186972 5219 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
I20260809 04:49:22.188771 5272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.188385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:39615 (local address 127.0.0.1:37436)
0809 04:49:22.188546 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.188645 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 04:49:22.188997 5270 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.60:39615: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.60.keytab.
Entry for principal kudu/127.12.234.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.60.keytab.
I20260809 04:49:22.213438 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.60:39615
--webserver_interface=127.12.234.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.60.keytab
--principal=kudu/127.12.234.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:38245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:39467,127.12.234.61:33601,127.12.234.60:39615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:49:22.238340 5270 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } attempt: 1
I20260809 04:49:22.239503 5272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.239064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:39615 (local address 127.0.0.1:37450)
0809 04:49:22.239249 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.239344 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 04:49:22.240494 5272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.240196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:39615 (local address 127.0.0.1:37454)
0809 04:49:22.240336 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.240404 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 04:49:22.240693 5270 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.60:39615: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
I20260809 04:49:22.308969 5270 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } attempt: 2
I20260809 04:49:22.310034 5272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.309593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:39615 (local address 127.0.0.1:37470)
0809 04:49:22.309780 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.309871 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 04:49:22.311048 5272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.310734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:39615 (local address 127.0.0.1:37484)
0809 04:49:22.310889 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.310951 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 04:49:22.311232 5270 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.60:39615: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
I20260809 04:49:22.353324 5194 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.61" port: 33601 } attempt: 5
Aug 09 04:49:22 dist-test-slave-1jjb krb5kdc[5109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250961, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for kudu/127.12.234.61@KRBTEST.COM
I20260809 04:49:22.367852 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.354117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:33601 (local address 127.0.0.1:38492)
0809 04:49:22.354330 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.354353 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:22.354479 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:22.354851 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:22.354854 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:22.355266 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:22.355450 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:22.355458 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.356957 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.356960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:22.357731 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:22.357740 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.358779 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.358782 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:22.358936 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:22.359552 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:22.359570 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:22.361795 (+ 2225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:22.365772 (+ 3977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:22.365776 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:22.365779 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:22.366146 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:22.366555 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:22.366558 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:22.366559 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:22.366667 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:22.367232 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:22.367235 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:22.367355 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 04:49:22.367455 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 04:49:22.367630 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 04:49:22.368093 5274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.354307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38492 (local address 127.12.234.61:33601)
0809 04:49:22.354505 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:22.354512 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:22.354535 (+ 23us) server_negotiation.cc:407] Connection header received
0809 04:49:22.354617 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:22.354623 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:22.354711 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:22.354821 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:22.355631 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.356776 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.357885 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.358642 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.358783 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:22.362014 (+ 3231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:22.362022 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:22.362029 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:22.362074 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:22.365632 (+ 3558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.366271 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.366277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.366280 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.366459 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.366782 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.366789 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.366791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.367117 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:22.367265 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:22.367585 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 04:49:22.367789 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 04:49:22.369177 5194 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:49:22.370167 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.369814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:39615 (local address 127.0.0.1:37496)
0809 04:49:22.369973 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.370042 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 04:49:22.371021 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.370744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:39615 (local address 127.0.0.1:37504)
0809 04:49:22.370868 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.370945 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 04:49:22.371233 5194 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.60:39615: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
I20260809 04:49:22.402441 5270 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } attempt: 3
I20260809 04:49:22.403504 5272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.403048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:39615 (local address 127.0.0.1:37520)
0809 04:49:22.403277 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.403354 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 04:49:22.404579 5272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.404208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:39615 (local address 127.0.0.1:37532)
0809 04:49:22.404418 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.404480 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260809 04:49:22.404798 5270 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.60:39615: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
I20260809 04:49:22.422415 5194 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } attempt: 1
I20260809 04:49:22.423431 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.423027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:39615 (local address 127.0.0.1:37538)
0809 04:49:22.423193 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.423282 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 04:49:22.424306 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.423980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:39615 (local address 127.0.0.1:37540)
0809 04:49:22.424141 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.424203 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 04:49:22.424491 5194 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.60:39615: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
I20260809 04:49:22.492693 5194 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } attempt: 2
I20260809 04:49:22.493824 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.493379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:39615 (local address 127.0.0.1:37556)
0809 04:49:22.493582 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.493673 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 04:49:22.494858 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.494480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:39615 (local address 127.0.0.1:37570)
0809 04:49:22.494642 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.494747 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 04:49:22.495091 5194 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.60:39615: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
W20260809 04:49:22.534076 5277 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 04:49:22.534413 5277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:22.534550 5277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:22.534685 5277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:22.541217 5277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:22.541317 5277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:22.541433 5277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:22.541519 5277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:22.541601 5277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:22.547986 5270 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } attempt: 4
I20260809 04:49:22.549077 5272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.548665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:39615 (local address 127.0.0.1:37576)
0809 04:49:22.548852 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.548932 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 04:49:22.550019 5272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.549715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:39615 (local address 127.0.0.1:37582)
0809 04:49:22.549853 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.549928 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 04:49:22.550204 5270 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.60:39615: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
I20260809 04:49:22.550953 5277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:38245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:39467,127.12.234.61:33601,127.12.234.60:39615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.60.keytab
--principal=kudu/127.12.234.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.60:39615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.234.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:49:22.552019 5277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:22.553265 5277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:22.564539 5282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:22.564522 5285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:22.564584 5283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:22.569410 5277 server_base.cc:1034] running on GCE node
I20260809 04:49:22.570187 5277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:22.571357 5277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:22.572598 5277 hybrid_clock.cc:648] HybridClock initialized: now 1786250962572544 us; error 53 us; skew 500 ppm
I20260809 04:49:22.572912 5277 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:49:22 dist-test-slave-1jjb krb5kdc[5109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250962, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:22.575865 5277 init.cc:377] Logged in from keytab as kudu/127.12.234.60@KRBTEST.COM (short username kudu)
I20260809 04:49:22.577337 5277 webserver.cc:460] Webserver started at http://127.12.234.60:43221/ using document root <none> and password file <none>
I20260809 04:49:22.577960 5277 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:22.578078 5277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:22.578433 5277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:22.581529 5277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-2/data/instance:
uuid: "2dc10ec8638b425aa571a78c1660ef52"
format_stamp: "Formatted at 2026-08-09 04:49:22 on dist-test-slave-1jjb"
server_key: "9707981525b8119e830bdf326ad13580"
server_key_iv: "8e57d9256cebe206cb2c341c726d8ad1"
server_key_version: "encryptionkey@0"
I20260809 04:49:22.582427 5277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-2/wal/instance:
uuid: "2dc10ec8638b425aa571a78c1660ef52"
format_stamp: "Formatted at 2026-08-09 04:49:22 on dist-test-slave-1jjb"
server_key: "9707981525b8119e830bdf326ad13580"
server_key_iv: "8e57d9256cebe206cb2c341c726d8ad1"
server_key_version: "encryptionkey@0"
I20260809 04:49:22.587054 5277 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260809 04:49:22.590148 5295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:22.591300 5277 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 04:49:22.591480 5277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-2/wal
uuid: "2dc10ec8638b425aa571a78c1660ef52"
format_stamp: "Formatted at 2026-08-09 04:49:22 on dist-test-slave-1jjb"
server_key: "9707981525b8119e830bdf326ad13580"
server_key_iv: "8e57d9256cebe206cb2c341c726d8ad1"
server_key_version: "encryptionkey@0"
I20260809 04:49:22.591674 5277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:22.601286 5194 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } attempt: 3
I20260809 04:49:22.602247 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.601814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:39615 (local address 127.0.0.1:37594)
0809 04:49:22.602016 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.602090 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 04:49:22.603178 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.602829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:39615 (local address 127.0.0.1:37602)
0809 04:49:22.603015 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.603084 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 04:49:22.603406 5194 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.60:39615: Network error: Client connection negotiation failed: client connection to 127.12.234.60:39615: connect: Connection refused (error 111)
I20260809 04:49:22.612300 5277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:22.616056 5277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:22.616379 5277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:22.634711 5277 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.60:39615
I20260809 04:49:22.634754 5349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.60:39615 every 8 connection(s)
I20260809 04:49:22.635883 5277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-2/data/info.pb
I20260809 04:49:22.639139 5350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:22.639281 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 5277
I20260809 04:49:22.639456 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786250603326224-13224-0/minicluster-data/master-2/wal/instance
I20260809 04:49:22.639815 13224 external_mini_cluster.cc:1428] Setting key bd2db23f0f923bb4a921f51840fb1faa
I20260809 04:49:22.647007 5350 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.62" port: 39467 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:49:22 dist-test-slave-1jjb krb5kdc[5109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.60@KRBTEST.COM
Aug 09 04:49:22 dist-test-slave-1jjb krb5kdc[5109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250962, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.60@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:49:22.661023 5353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.641477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37616 (local address 127.12.234.60:39615)
0809 04:49:22.642183 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:22.642193 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:22.642233 (+ 40us) server_negotiation.cc:407] Connection header received
0809 04:49:22.643680 (+ 1447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:22.643698 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:22.644038 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:22.644505 (+ 467us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:22.645886 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.646951 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.648129 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.648673 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.648842 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:22.653740 (+ 4898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:22.653747 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:22.653757 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:22.653788 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:22.657159 (+ 3371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.657938 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.657945 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.657952 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.658112 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.658468 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.658473 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.658476 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.659002 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:22.659283 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:22.659689 (+ 406us) server_negotiation.cc:299] Negotiation successful
0809 04:49:22.659998 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":300,"threads_started":1}
I20260809 04:49:22.664408 5354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.649083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:39467 (local address 127.0.0.1:45292)
0809 04:49:22.649860 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.649908 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:22.650046 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:22.650396 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:22.650404 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:22.651173 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:22.651477 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:22.651490 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.652536 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.652544 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:22.654034 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:22.654046 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.655041 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.655046 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:22.655153 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:22.655739 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:22.655772 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:22.657895 (+ 2123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:22.661530 (+ 3635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:22.661541 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:22.661547 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:22.661911 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:22.662417 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:22.662422 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:22.662426 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:22.662676 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:22.663459 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:22.663465 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:22.663635 (+ 170us) client_negotiation.cc:769] Sending connection context
0809 04:49:22.663797 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 04:49:22.664096 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":346,"threads_started":1}
I20260809 04:49:22.665217 5196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.649114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45292 (local address 127.12.234.62:39467)
0809 04:49:22.649357 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:22.649364 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:22.649997 (+ 633us) server_negotiation.cc:407] Connection header received
0809 04:49:22.650157 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:22.650160 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:22.650215 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:22.650287 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:22.651648 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.652381 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.654174 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.654866 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.654982 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:22.658050 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:22.658054 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:22.658057 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:22.658085 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:22.661408 (+ 3323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.662039 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.662054 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.662056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.662280 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.662813 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.662818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.662821 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.663171 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:22.663356 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:22.664836 (+ 1480us) server_negotiation.cc:299] Negotiation successful
0809 04:49:22.664982 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":70}
I20260809 04:49:22.666306 5350 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.61" port: 33601 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:49:22 dist-test-slave-1jjb krb5kdc[5109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
Aug 09 04:49:22 dist-test-slave-1jjb krb5kdc[5109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250962, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.60@KRBTEST.COM for kudu/127.12.234.61@KRBTEST.COM
I20260809 04:49:22.679092 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.663907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45296 (local address 127.12.234.62:39467)
0809 04:49:22.664494 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:22.664498 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:22.664517 (+ 19us) server_negotiation.cc:407] Connection header received
0809 04:49:22.664554 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:22.664558 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:22.664609 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:22.664703 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:22.666242 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.667095 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.668439 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.668759 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.668918 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:22.673379 (+ 4461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:22.673386 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:22.673391 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:22.673439 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:22.676579 (+ 3140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.677152 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.677157 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.677161 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.677317 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.677724 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.677727 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.677729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.678001 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:22.678259 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:22.678568 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 04:49:22.678756 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":317,"threads_started":1}
I20260809 04:49:22.679733 5354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.666917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:33601 (local address 127.0.0.1:38496)
0809 04:49:22.667068 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.667083 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:22.667166 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:22.667465 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:22.667468 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:22.667716 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:22.667835 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:22.667842 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.668881 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.668886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:22.669938 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:22.669950 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.670884 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.670888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:22.671006 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:22.671519 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:22.671534 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:22.673947 (+ 2413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:22.677399 (+ 3452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:22.677404 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:22.677408 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:22.677882 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:22.678270 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:22.678274 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:22.678278 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:22.678434 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:22.678964 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:22.678967 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:22.679077 (+ 110us) client_negotiation.cc:769] Sending connection context
0809 04:49:22.679208 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 04:49:22.679454 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 04:49:22.680207 5274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.666957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38496 (local address 127.12.234.61:33601)
0809 04:49:22.667122 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:22.667125 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:22.667139 (+ 14us) server_negotiation.cc:407] Connection header received
0809 04:49:22.667276 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:22.667281 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:22.667350 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:22.667443 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:22.667980 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.668736 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.670080 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.670754 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.670902 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:22.674202 (+ 3300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:22.674210 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:22.674217 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:22.674258 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:22.677245 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.677990 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.677996 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.677998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.678157 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.678544 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.678548 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.678551 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.678836 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:22.678977 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:22.679600 (+ 623us) server_negotiation.cc:299] Negotiation successful
0809 04:49:22.679809 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 04:49:22.681134 5350 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:49:22 dist-test-slave-1jjb krb5kdc[5109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250962, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.60@KRBTEST.COM for kudu/127.12.234.60@KRBTEST.COM
Aug 09 04:49:22 dist-test-slave-1jjb krb5kdc[5109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.61@KRBTEST.COM
I20260809 04:49:22.693478 5354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.681721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:39615 (local address 127.0.0.1:37622)
0809 04:49:22.681887 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.681904 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:22.681992 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:22.682334 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:22.682349 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:22.682725 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:22.682885 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:22.682895 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.683839 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.683843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:22.684624 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:22.684636 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.685315 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.685319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:22.685466 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:22.686042 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:22.686056 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:22.687637 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:22.690944 (+ 3307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:22.690952 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:22.690955 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:22.691388 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:22.691718 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:22.691723 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:22.691727 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:22.691888 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:22.692476 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:22.692481 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:22.692648 (+ 167us) client_negotiation.cc:769] Sending connection context
0809 04:49:22.692812 (+ 164us) client_negotiation.cc:240] Negotiation successful
0809 04:49:22.693111 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 04:49:22.693697 5353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.681955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37622 (local address 127.12.234.60:39615)
0809 04:49:22.682099 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:22.682103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:22.682116 (+ 13us) server_negotiation.cc:407] Connection header received
0809 04:49:22.682176 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:22.682179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:22.682223 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:22.682318 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:22.683022 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.683729 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.684745 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.685158 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.685280 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:22.687810 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:22.687814 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:22.687816 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:22.687841 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:22.690821 (+ 2980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.691503 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.691507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.691508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.691605 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.691988 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.691991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.691993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.692328 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:22.692463 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:22.692930 (+ 467us) server_negotiation.cc:299] Negotiation successful
0809 04:49:22.693342 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":125,"server-negotiator.queue_time_us":52}
I20260809 04:49:22.695012 5274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.681051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38506 (local address 127.12.234.61:33601)
0809 04:49:22.681193 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:22.681200 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:22.681217 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:49:22.681260 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:22.681264 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:22.681321 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:22.681438 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:22.682725 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.683502 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.684633 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.685078 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.685158 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:22.689257 (+ 4099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:22.689261 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:22.689264 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:22.689292 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:22.692073 (+ 2781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.692905 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.692911 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.692915 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.693096 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.693565 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.693570 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.693573 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.693899 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:22.694163 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:22.694443 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 04:49:22.694703 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 04:49:22.698611 5350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dc10ec8638b425aa571a78c1660ef52: Bootstrap starting.
I20260809 04:49:22.699167 13224 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:22.701490 5350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2dc10ec8638b425aa571a78c1660ef52: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:22.702466 5350 log.cc:824] T 00000000000000000000000000000000 P 2dc10ec8638b425aa571a78c1660ef52: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:22.701502 5166 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "938a8c77fa414a5ab9462d7f442eec2c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45296
I20260809 04:49:22.705053 5350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dc10ec8638b425aa571a78c1660ef52: No bootstrap required, opened a new log
I20260809 04:49:22.711221 5350 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2dc10ec8638b425aa571a78c1660ef52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "938a8c77fa414a5ab9462d7f442eec2c" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 39467 } } peers { permanent_uuid: "d953ba5fa0a749eeb1c6c1fa58a7a0e4" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 33601 } } peers { permanent_uuid: "2dc10ec8638b425aa571a78c1660ef52" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } }
I20260809 04:49:22.711519 5350 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2dc10ec8638b425aa571a78c1660ef52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dc10ec8638b425aa571a78c1660ef52, State: Initialized, Role: FOLLOWER
I20260809 04:49:22.711927 5350 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2dc10ec8638b425aa571a78c1660ef52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "938a8c77fa414a5ab9462d7f442eec2c" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 39467 } } peers { permanent_uuid: "d953ba5fa0a749eeb1c6c1fa58a7a0e4" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 33601 } } peers { permanent_uuid: "2dc10ec8638b425aa571a78c1660ef52" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } }
I20260809 04:49:22.712872 5357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dc10ec8638b425aa571a78c1660ef52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "938a8c77fa414a5ab9462d7f442eec2c" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 39467 } } peers { permanent_uuid: "d953ba5fa0a749eeb1c6c1fa58a7a0e4" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 33601 } } peers { permanent_uuid: "2dc10ec8638b425aa571a78c1660ef52" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } } }
I20260809 04:49:22.713232 5357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dc10ec8638b425aa571a78c1660ef52 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:22.713853 5350 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2dc10ec8638b425aa571a78c1660ef52 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 04:49:22.722429 5369 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2dc10ec8638b425aa571a78c1660ef52: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:49:22.722541 5369 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:49:22.720248 5277 master_runner.cc:186] Error getting master registration for 127.12.234.62:39467: OK, instance_id { permanent_uuid: "938a8c77fa414a5ab9462d7f442eec2c" instance_seqno: 1786250961651139 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 04:49:22.727563 5277 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 04:49:22.781298 5194 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } attempt: 4
Aug 09 04:49:22 dist-test-slave-1jjb krb5kdc[5109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250961, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for kudu/127.12.234.60@KRBTEST.COM
I20260809 04:49:22.798162 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.782195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:39615 (local address 127.0.0.1:37628)
0809 04:49:22.782391 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.782412 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:22.782573 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:22.782944 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:22.782951 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:22.783373 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:22.783534 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:22.783543 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.785038 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.785045 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:22.786515 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:22.786531 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.787522 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.787528 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:22.787729 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:22.788490 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:22.788513 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:22.791543 (+ 3030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:22.795537 (+ 3994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:22.795542 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:22.795545 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:22.795997 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:22.796392 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:22.796396 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:22.796400 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:22.796556 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:22.797152 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:22.797158 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:22.797306 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 04:49:22.797474 (+ 168us) client_negotiation.cc:240] Negotiation successful
0809 04:49:22.797759 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 04:49:22.798182 5353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.782100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37628 (local address 127.12.234.60:39615)
0809 04:49:22.782389 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:22.782395 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:22.782497 (+ 102us) server_negotiation.cc:407] Connection header received
0809 04:49:22.782708 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:22.782713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:22.782799 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:22.782904 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:22.783681 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.784864 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.786672 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.787357 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.787476 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:22.791729 (+ 4253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:22.791736 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:22.791740 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:22.791779 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:22.795425 (+ 3646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.796135 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.796141 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.796146 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.796286 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.796658 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.796663 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.796668 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.797014 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:22.797158 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:22.797579 (+ 421us) server_negotiation.cc:299] Negotiation successful
0809 04:49:22.797795 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260809 04:49:22.804200 5194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c: Bootstrap starting.
I20260809 04:49:22.807926 5194 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:22.809122 5194 log.cc:824] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:22.812593 5194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c: No bootstrap required, opened a new log
I20260809 04:49:22.820158 5194 raft_consensus.cc:348] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "938a8c77fa414a5ab9462d7f442eec2c" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 39467 } } peers { permanent_uuid: "d953ba5fa0a749eeb1c6c1fa58a7a0e4" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 33601 } } peers { permanent_uuid: "2dc10ec8638b425aa571a78c1660ef52" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } }
I20260809 04:49:22.820574 5194 raft_consensus.cc:729] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 938a8c77fa414a5ab9462d7f442eec2c, State: Initialized, Role: FOLLOWER
I20260809 04:49:22.821070 5194 consensus_queue.cc:260] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "938a8c77fa414a5ab9462d7f442eec2c" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 39467 } } peers { permanent_uuid: "d953ba5fa0a749eeb1c6c1fa58a7a0e4" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 33601 } } peers { permanent_uuid: "2dc10ec8638b425aa571a78c1660ef52" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } }
I20260809 04:49:22.822036 5372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "938a8c77fa414a5ab9462d7f442eec2c" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 39467 } } peers { permanent_uuid: "d953ba5fa0a749eeb1c6c1fa58a7a0e4" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 33601 } } peers { permanent_uuid: "2dc10ec8638b425aa571a78c1660ef52" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } } }
I20260809 04:49:22.822428 5372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:22.823361 5194 sys_catalog.cc:564] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:22.823426 5166 raft_consensus.cc:482] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 04:49:22.823693 5166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:22.826314 5166 raft_consensus.cc:504] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "938a8c77fa414a5ab9462d7f442eec2c" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 39467 } } peers { permanent_uuid: "d953ba5fa0a749eeb1c6c1fa58a7a0e4" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 33601 } } peers { permanent_uuid: "2dc10ec8638b425aa571a78c1660ef52" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } }
I20260809 04:49:22.827499 5166 leader_election.cc:288] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c [CANDIDATE]: Term 1 election: Requested vote from peers d953ba5fa0a749eeb1c6c1fa58a7a0e4 (127.12.234.61:33601), 2dc10ec8638b425aa571a78c1660ef52 (127.12.234.60:39615)
I20260809 04:49:22.828899 5323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "938a8c77fa414a5ab9462d7f442eec2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2dc10ec8638b425aa571a78c1660ef52"
I20260809 04:49:22.829398 5323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2dc10ec8638b425aa571a78c1660ef52 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:22.832595 5323 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2dc10ec8638b425aa571a78c1660ef52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 938a8c77fa414a5ab9462d7f442eec2c in term 1.
I20260809 04:49:22.833568 5139 leader_election.cc:302] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dc10ec8638b425aa571a78c1660ef52, 938a8c77fa414a5ab9462d7f442eec2c; no voters:
I20260809 04:49:22.834012 5372 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:22.845108 5372 raft_consensus.cc:686] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c [term 1 LEADER]: Becoming Leader. State: Replica: 938a8c77fa414a5ab9462d7f442eec2c, State: Running, Role: LEADER
I20260809 04:49:22.845815 5372 consensus_queue.cc:237] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "938a8c77fa414a5ab9462d7f442eec2c" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 39467 } } peers { permanent_uuid: "d953ba5fa0a749eeb1c6c1fa58a7a0e4" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 33601 } } peers { permanent_uuid: "2dc10ec8638b425aa571a78c1660ef52" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } }
I20260809 04:49:22.849397 5270 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } attempt: 5
Aug 09 04:49:22 dist-test-slave-1jjb krb5kdc[5109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250962, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.61@KRBTEST.COM for kudu/127.12.234.60@KRBTEST.COM
I20260809 04:49:22.862752 5380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 938a8c77fa414a5ab9462d7f442eec2c. Latest consensus state: current_term: 1 leader_uuid: "938a8c77fa414a5ab9462d7f442eec2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "938a8c77fa414a5ab9462d7f442eec2c" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 39467 } } peers { permanent_uuid: "d953ba5fa0a749eeb1c6c1fa58a7a0e4" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 33601 } } peers { permanent_uuid: "2dc10ec8638b425aa571a78c1660ef52" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } } }
I20260809 04:49:22.863116 5380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c [sys.catalog]: This master's current role is: LEADER
W20260809 04:49:22.868199 5301 tablet.cc:2369] T 00000000000000000000000000000000 P 2dc10ec8638b425aa571a78c1660ef52: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:49:22.869922 5272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.850083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:39615 (local address 127.0.0.1:37638)
0809 04:49:22.850530 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:22.850553 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:22.850679 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:22.851606 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:22.851613 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:22.852283 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:22.852517 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:22.852536 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.854422 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.854431 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:22.856259 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:22.856275 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.857677 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.857701 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:22.858023 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:22.858774 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:22.858803 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:22.862119 (+ 3316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:22.866775 (+ 4656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:22.866780 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:22.866783 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:22.867289 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:22.867752 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:22.867758 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:22.867763 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:22.867941 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:22.868799 (+ 858us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:22.868804 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:22.868994 (+ 190us) client_negotiation.cc:769] Sending connection context
0809 04:49:22.869189 (+ 195us) client_negotiation.cc:240] Negotiation successful
0809 04:49:22.869494 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297}
I20260809 04:49:22.871114 5353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.850912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37638 (local address 127.12.234.60:39615)
0809 04:49:22.851109 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:22.851115 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:22.851139 (+ 24us) server_negotiation.cc:407] Connection header received
0809 04:49:22.851200 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:22.851211 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:22.851296 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:22.851396 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:22.852721 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.854232 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.856475 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.857270 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.857418 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:22.862372 (+ 4954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:22.862381 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:22.862387 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:22.862432 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:22.866597 (+ 4165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.867433 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.867439 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.867444 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.867603 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.868111 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.868122 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.868128 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.868578 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:22.868733 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:22.870375 (+ 1642us) server_negotiation.cc:299] Negotiation successful
0809 04:49:22.870651 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 04:49:22.877912 5270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d953ba5fa0a749eeb1c6c1fa58a7a0e4: Bootstrap starting.
I20260809 04:49:22.882478 5270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d953ba5fa0a749eeb1c6c1fa58a7a0e4: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:22.883937 5270 log.cc:824] T 00000000000000000000000000000000 P d953ba5fa0a749eeb1c6c1fa58a7a0e4: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:22.885106 5389 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260809 04:49:22.891808 5144 tablet.cc:2369] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:49:22.894021 5270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d953ba5fa0a749eeb1c6c1fa58a7a0e4: No bootstrap required, opened a new log
I20260809 04:49:22.907023 5389 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 04:49:22.890784 5122 master_runner.cc:186] Error getting master registration for 127.12.234.61:33601: OK, instance_id { permanent_uuid: "d953ba5fa0a749eeb1c6c1fa58a7a0e4" instance_seqno: 1786250962129025 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 04:49:22.908983 5122 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 04:49:22.914582 5274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.870160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38518 (local address 127.12.234.61:33601)
0809 04:49:22.870319 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:22.870324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:22.886483 (+ 16159us) server_negotiation.cc:407] Connection header received
0809 04:49:22.886683 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:22.886688 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:22.886749 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:22.886808 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:22.888895 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.890087 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.891992 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.892681 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.892777 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:22.896164 (+ 3387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:22.896171 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:22.896176 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:22.896216 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:22.899954 (+ 3738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.904944 (+ 4990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.904949 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.904952 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.911137 (+ 6185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.912461 (+ 1324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.912469 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.912472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.912819 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:22.913042 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:22.914020 (+ 978us) server_negotiation.cc:299] Negotiation successful
0809 04:49:22.914242 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:49:22.919770 5270 raft_consensus.cc:348] T 00000000000000000000000000000000 P d953ba5fa0a749eeb1c6c1fa58a7a0e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "938a8c77fa414a5ab9462d7f442eec2c" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 39467 } } peers { permanent_uuid: "d953ba5fa0a749eeb1c6c1fa58a7a0e4" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 33601 } } peers { permanent_uuid: "2dc10ec8638b425aa571a78c1660ef52" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } }
I20260809 04:49:22.920235 5270 raft_consensus.cc:729] T 00000000000000000000000000000000 P d953ba5fa0a749eeb1c6c1fa58a7a0e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d953ba5fa0a749eeb1c6c1fa58a7a0e4, State: Initialized, Role: FOLLOWER
I20260809 04:49:22.920763 5270 consensus_queue.cc:260] T 00000000000000000000000000000000 P d953ba5fa0a749eeb1c6c1fa58a7a0e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "938a8c77fa414a5ab9462d7f442eec2c" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 39467 } } peers { permanent_uuid: "d953ba5fa0a749eeb1c6c1fa58a7a0e4" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 33601 } } peers { permanent_uuid: "2dc10ec8638b425aa571a78c1660ef52" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } }
I20260809 04:49:22.925483 5323 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2dc10ec8638b425aa571a78c1660ef52 [term 1 FOLLOWER]: Refusing update from remote peer 938a8c77fa414a5ab9462d7f442eec2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:49:22.926491 5380 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2dc10ec8638b425aa571a78c1660ef52" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:49:22.946045 5355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.866972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45306 (local address 127.12.234.62:39467)
0809 04:49:22.867147 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:22.867154 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:22.886259 (+ 19105us) server_negotiation.cc:407] Connection header received
0809 04:49:22.886332 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:22.886336 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:22.886412 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:22.886530 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:22.903262 (+ 16732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.904331 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.909508 (+ 5177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.910206 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.910334 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:22.928748 (+ 18414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:22.928756 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:22.928760 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:22.928793 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:22.933465 (+ 4672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.939129 (+ 5664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.939173 (+ 44us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.939179 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.939532 (+ 353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.941074 (+ 1542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.941081 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.941118 (+ 37us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.942053 (+ 935us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:22.942559 (+ 506us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:22.944520 (+ 1961us) server_negotiation.cc:299] Negotiation successful
0809 04:49:22.945098 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 04:49:22.948575 5397 sys_catalog.cc:455] T 00000000000000000000000000000000 P d953ba5fa0a749eeb1c6c1fa58a7a0e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "938a8c77fa414a5ab9462d7f442eec2c" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 39467 } } peers { permanent_uuid: "d953ba5fa0a749eeb1c6c1fa58a7a0e4" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 33601 } } peers { permanent_uuid: "2dc10ec8638b425aa571a78c1660ef52" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } } }
I20260809 04:49:22.950716 5397 sys_catalog.cc:458] T 00000000000000000000000000000000 P d953ba5fa0a749eeb1c6c1fa58a7a0e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:22.950688 5245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "938a8c77fa414a5ab9462d7f442eec2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d953ba5fa0a749eeb1c6c1fa58a7a0e4"
I20260809 04:49:22.951108 5245 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d953ba5fa0a749eeb1c6c1fa58a7a0e4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:22.952348 5357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dc10ec8638b425aa571a78c1660ef52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 938a8c77fa414a5ab9462d7f442eec2c. Latest consensus state: current_term: 1 leader_uuid: "938a8c77fa414a5ab9462d7f442eec2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "938a8c77fa414a5ab9462d7f442eec2c" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 39467 } } peers { permanent_uuid: "d953ba5fa0a749eeb1c6c1fa58a7a0e4" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 33601 } } peers { permanent_uuid: "2dc10ec8638b425aa571a78c1660ef52" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } } }
I20260809 04:49:22.952656 5357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dc10ec8638b425aa571a78c1660ef52 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:22.950599 5270 sys_catalog.cc:564] T 00000000000000000000000000000000 P d953ba5fa0a749eeb1c6c1fa58a7a0e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:22.954531 5245 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d953ba5fa0a749eeb1c6c1fa58a7a0e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 938a8c77fa414a5ab9462d7f442eec2c in term 1.
I20260809 04:49:22.955479 5244 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d953ba5fa0a749eeb1c6c1fa58a7a0e4 [term 1 FOLLOWER]: Refusing update from remote peer 938a8c77fa414a5ab9462d7f442eec2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:49:22.956284 5380 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d953ba5fa0a749eeb1c6c1fa58a7a0e4" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 33601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:49:22.962745 5396 mvcc.cc:205] Tried to move back new op lower bound from 7316483944128606208 to 7316483943822852096. Current Snapshot: MvccSnapshot[applied={T|T < 7316483944128606208}]
I20260809 04:49:22.963670 5353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:22.900818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37646 (local address 127.12.234.60:39615)
0809 04:49:22.901054 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:22.901059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:22.915916 (+ 14857us) server_negotiation.cc:407] Connection header received
0809 04:49:22.916373 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:22.916378 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:22.916459 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:22.916524 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:22.921007 (+ 4483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.922364 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.933676 (+ 11312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:22.934547 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:22.934672 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:22.956208 (+ 21536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:22.956217 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:22.956221 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:22.956265 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:22.960307 (+ 4042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.961100 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.961105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.961108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.961238 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:22.961665 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:22.961669 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:22.961672 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:22.962043 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:22.962284 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:22.962841 (+ 557us) server_negotiation.cc:299] Negotiation successful
0809 04:49:22.963146 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 04:49:22.968119 5357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dc10ec8638b425aa571a78c1660ef52 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "938a8c77fa414a5ab9462d7f442eec2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "938a8c77fa414a5ab9462d7f442eec2c" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 39467 } } peers { permanent_uuid: "d953ba5fa0a749eeb1c6c1fa58a7a0e4" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 33601 } } peers { permanent_uuid: "2dc10ec8638b425aa571a78c1660ef52" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } } }
I20260809 04:49:22.968374 5357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dc10ec8638b425aa571a78c1660ef52 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:22.970054 5380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "938a8c77fa414a5ab9462d7f442eec2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "938a8c77fa414a5ab9462d7f442eec2c" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 39467 } } peers { permanent_uuid: "d953ba5fa0a749eeb1c6c1fa58a7a0e4" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 33601 } } peers { permanent_uuid: "2dc10ec8638b425aa571a78c1660ef52" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } } }
I20260809 04:49:22.970283 5380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:22.972434 5389 catalog_manager.cc:1347] Generated new cluster ID: 800ba18f03ca458792c6915da36063b5
I20260809 04:49:22.972533 5389 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:22.973052 5397 sys_catalog.cc:455] T 00000000000000000000000000000000 P d953ba5fa0a749eeb1c6c1fa58a7a0e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 938a8c77fa414a5ab9462d7f442eec2c. Latest consensus state: current_term: 1 leader_uuid: "938a8c77fa414a5ab9462d7f442eec2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "938a8c77fa414a5ab9462d7f442eec2c" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 39467 } } peers { permanent_uuid: "d953ba5fa0a749eeb1c6c1fa58a7a0e4" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 33601 } } peers { permanent_uuid: "2dc10ec8638b425aa571a78c1660ef52" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } } }
I20260809 04:49:22.973434 5397 sys_catalog.cc:458] T 00000000000000000000000000000000 P d953ba5fa0a749eeb1c6c1fa58a7a0e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:22.983402 5400 mvcc.cc:205] Tried to move back new op lower bound from 7316483944128606208 to 7316483943822852096. Current Snapshot: MvccSnapshot[applied={T|T < 7316483944128606208}]
I20260809 04:49:22.985504 5406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "938a8c77fa414a5ab9462d7f442eec2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "938a8c77fa414a5ab9462d7f442eec2c" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 39467 } } peers { permanent_uuid: "d953ba5fa0a749eeb1c6c1fa58a7a0e4" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 33601 } } peers { permanent_uuid: "2dc10ec8638b425aa571a78c1660ef52" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } } }
I20260809 04:49:22.985894 5406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:22.997798 5397 sys_catalog.cc:455] T 00000000000000000000000000000000 P d953ba5fa0a749eeb1c6c1fa58a7a0e4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "938a8c77fa414a5ab9462d7f442eec2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "938a8c77fa414a5ab9462d7f442eec2c" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 39467 } } peers { permanent_uuid: "d953ba5fa0a749eeb1c6c1fa58a7a0e4" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 33601 } } peers { permanent_uuid: "2dc10ec8638b425aa571a78c1660ef52" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 39615 } } }
I20260809 04:49:22.998049 5397 sys_catalog.cc:458] T 00000000000000000000000000000000 P d953ba5fa0a749eeb1c6c1fa58a7a0e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:23.000420 5389 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:23.002101 5389 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:23.025158 5389 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 938a8c77fa414a5ab9462d7f442eec2c: Generated new TSK 0
I20260809 04:49:23.026077 5389 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:23.051265 5416 catalog_manager.cc:1259] Loaded cluster ID: 800ba18f03ca458792c6915da36063b5
I20260809 04:49:23.051389 5416 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d953ba5fa0a749eeb1c6c1fa58a7a0e4: loading cluster ID for follower catalog manager: success
I20260809 04:49:23.055064 5416 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d953ba5fa0a749eeb1c6c1fa58a7a0e4: acquiring CA information for follower catalog manager: success
I20260809 04:49:23.057265 5416 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d953ba5fa0a749eeb1c6c1fa58a7a0e4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 04:49:23.168291 5355 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:23.147642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45314 (local address 127.12.234.62:39467)
0809 04:49:23.147809 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:23.147816 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:23.155047 (+ 7231us) server_negotiation.cc:407] Connection header received
0809 04:49:23.155336 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:23.155355 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:23.155438 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:23.155515 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:23.168005 (+ 12490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45314: BlockingRecv error: recv got EOF from 127.0.0.1:45314 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260809 04:49:23.168439 5274 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:23.149086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38520 (local address 127.12.234.61:33601)
0809 04:49:23.149324 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:23.149332 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:23.164572 (+ 15240us) server_negotiation.cc:407] Connection header received
0809 04:49:23.164825 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:23.164831 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:23.164926 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:23.164993 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:23.168230 (+ 3237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38520: BlockingRecv error: recv got EOF from 127.0.0.1:38520 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260809 04:49:23.177896 5353 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:23.155266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37658 (local address 127.12.234.60:39615)
0809 04:49:23.155461 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:23.155467 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:23.166210 (+ 10743us) server_negotiation.cc:407] Connection header received
0809 04:49:23.166539 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:23.166545 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:23.166642 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:23.166705 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:23.177671 (+ 10966us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37658: BlockingRecv error: recv got EOF from 127.0.0.1:37658 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 04:49:23.235869 5274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:23.220599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38524 (local address 127.12.234.61:33601)
0809 04:49:23.220781 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:23.220788 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:23.228558 (+ 7770us) server_negotiation.cc:407] Connection header received
0809 04:49:23.228828 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:23.228834 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:23.229028 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:23.229242 (+ 214us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:23.229941 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.231069 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.233071 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.233744 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.233837 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:23.234518 (+ 681us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:23.235109 (+ 591us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:23.235225 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:23.235459 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 04:49:23.235546 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 04:49:23.249130 5353 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:23.247493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37670 (local address 127.12.234.60:39615)
0809 04:49:23.247637 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:23.247641 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:23.247726 (+ 85us) server_negotiation.cc:407] Connection header received
0809 04:49:23.247913 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:23.247918 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:23.248025 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:23.248159 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:23.248899 (+ 740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37670: BlockingRecv error: recv got EOF from 127.0.0.1:37670 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 04:49:23.258888 5274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:23.253290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38532 (local address 127.12.234.61:33601)
0809 04:49:23.253509 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:23.253515 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:23.253531 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:49:23.253685 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:23.253690 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:23.253782 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:23.253837 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:23.254375 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.255391 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.257105 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.257718 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.257827 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:23.258210 (+ 383us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:23.258332 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:23.258415 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:23.258551 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 04:49:23.258618 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 04:49:23.271648 5353 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:23.270252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37682 (local address 127.12.234.60:39615)
0809 04:49:23.270420 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:23.270425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:23.270442 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:49:23.270499 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:23.270504 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:23.270586 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:23.270681 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:23.271458 (+ 777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37682: BlockingRecv error: recv got EOF from 127.0.0.1:37682 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:49:23.282886 5274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:23.277095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38536 (local address 127.12.234.61:33601)
0809 04:49:23.277244 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:23.277249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:23.277361 (+ 112us) server_negotiation.cc:407] Connection header received
0809 04:49:23.277605 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:23.277610 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:23.277675 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:23.277729 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:23.278276 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.279214 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.280888 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.281495 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.281573 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:23.282089 (+ 516us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:23.282218 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:23.282272 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:23.282488 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 04:49:23.282567 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260809 04:49:23.295521 5353 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:23.293260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37696 (local address 127.12.234.60:39615)
0809 04:49:23.293454 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:23.293459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:23.293892 (+ 433us) server_negotiation.cc:407] Connection header received
0809 04:49:23.294115 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:23.294119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:23.294199 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:23.294282 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:23.295257 (+ 975us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37696: BlockingRecv error: recv got EOF from 127.0.0.1:37696 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 04:49:23.310235 5274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:23.303131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38538 (local address 127.12.234.61:33601)
0809 04:49:23.303346 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:23.303353 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:23.303822 (+ 469us) server_negotiation.cc:407] Connection header received
0809 04:49:23.304091 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:23.304097 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:23.304178 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:23.304280 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:23.304913 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.306111 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.308069 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.308760 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.308845 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:23.309477 (+ 632us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:23.309604 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:23.309666 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:23.309900 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 04:49:23.309986 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
W20260809 04:49:23.323663 5353 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:23.321134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37706 (local address 127.12.234.60:39615)
0809 04:49:23.321425 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:23.321434 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:23.321966 (+ 532us) server_negotiation.cc:407] Connection header received
0809 04:49:23.322240 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:23.322246 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:23.322339 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:23.322420 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:23.323378 (+ 958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37706: BlockingRecv error: recv got EOF from 127.0.0.1:37706 (error 108)
Metrics: {"server-negotiator.queue_time_us":126}
I20260809 04:49:23.344602 5274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:23.337753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38544 (local address 127.12.234.61:33601)
0809 04:49:23.337941 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:23.337947 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:23.338359 (+ 412us) server_negotiation.cc:407] Connection header received
0809 04:49:23.338564 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:23.338572 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:23.338667 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:23.338780 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:23.339255 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.340428 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.342375 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.343133 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.343264 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:23.343728 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:23.343873 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:23.343968 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:23.344167 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 04:49:23.344252 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 04:49:23.356513 5353 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:23.354811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37714 (local address 127.12.234.60:39615)
0809 04:49:23.354962 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:23.354967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:23.354985 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:49:23.355175 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:23.355180 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:23.355254 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:23.355309 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:23.356295 (+ 986us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37714: BlockingRecv error: recv got EOF from 127.0.0.1:37714 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 04:49:23.382768 5274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:23.375854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38558 (local address 127.12.234.61:33601)
0809 04:49:23.376031 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:23.376038 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:23.376543 (+ 505us) server_negotiation.cc:407] Connection header received
0809 04:49:23.376738 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:23.376744 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:23.376830 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:23.376934 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:23.377410 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.378623 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.380424 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.381027 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.381145 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:23.381597 (+ 452us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:23.382086 (+ 489us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:23.382194 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:23.382382 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 04:49:23.382483 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 04:49:23.395529 5353 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:23.393729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37722 (local address 127.12.234.60:39615)
0809 04:49:23.393940 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:23.393946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:23.393969 (+ 23us) server_negotiation.cc:407] Connection header received
0809 04:49:23.394256 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:23.394262 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:23.394355 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:23.394473 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:23.395314 (+ 841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37722: BlockingRecv error: recv got EOF from 127.0.0.1:37722 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 04:49:23.444725 5274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:23.438107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38560 (local address 127.12.234.61:33601)
0809 04:49:23.438330 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:23.438336 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:23.438445 (+ 109us) server_negotiation.cc:407] Connection header received
0809 04:49:23.438749 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:23.438755 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:23.438860 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:23.438953 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:23.439475 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.440647 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.442489 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.443252 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.443381 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:23.443894 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:23.444029 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:23.444134 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:23.444310 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 04:49:23.444400 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 04:49:23.453622 5353 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:23.451725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37726 (local address 127.12.234.60:39615)
0809 04:49:23.451888 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:23.451893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:23.451975 (+ 82us) server_negotiation.cc:407] Connection header received
0809 04:49:23.452275 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:23.452281 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:23.452370 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:23.452476 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:23.453365 (+ 889us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37726: BlockingRecv error: recv got EOF from 127.0.0.1:37726 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 04:49:23.526824 5274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:23.521265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38576 (local address 127.12.234.61:33601)
0809 04:49:23.521514 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:23.521520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:23.521873 (+ 353us) server_negotiation.cc:407] Connection header received
0809 04:49:23.522052 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:23.522055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:23.522117 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:23.522202 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:23.522668 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.523556 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.525241 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.525609 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.525692 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:23.526132 (+ 440us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:23.526220 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:23.526279 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:23.526473 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 04:49:23.526532 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 04:49:23.534284 5353 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:23.532554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37732 (local address 127.12.234.60:39615)
0809 04:49:23.532751 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:23.532756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:23.533181 (+ 425us) server_negotiation.cc:407] Connection header received
0809 04:49:23.533331 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:23.533334 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:23.533405 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:23.533492 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:23.534102 (+ 610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37732: BlockingRecv error: recv got EOF from 127.0.0.1:37732 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 04:49:23.670440 5274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:23.665484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38590 (local address 127.12.234.61:33601)
0809 04:49:23.665759 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:23.665767 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:23.666060 (+ 293us) server_negotiation.cc:407] Connection header received
0809 04:49:23.666274 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:23.666278 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:23.666341 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:23.666431 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:23.666834 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.667460 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.668819 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.669237 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.669378 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:23.669746 (+ 368us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:23.669861 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:23.669961 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:23.670103 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 04:49:23.670182 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260809 04:49:23.677170 5353 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:23.675359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37746 (local address 127.12.234.60:39615)
0809 04:49:23.675497 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:23.675508 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:23.675877 (+ 369us) server_negotiation.cc:407] Connection header received
0809 04:49:23.676070 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:23.676118 (+ 48us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:23.676199 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:23.676280 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:23.676958 (+ 678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37746: BlockingRecv error: recv got EOF from 127.0.0.1:37746 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 04:49:23.724905 5369 catalog_manager.cc:1259] Loaded cluster ID: 800ba18f03ca458792c6915da36063b5
I20260809 04:49:23.725014 5369 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2dc10ec8638b425aa571a78c1660ef52: loading cluster ID for follower catalog manager: success
I20260809 04:49:23.727355 5369 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2dc10ec8638b425aa571a78c1660ef52: acquiring CA information for follower catalog manager: success
I20260809 04:49:23.729156 5369 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2dc10ec8638b425aa571a78c1660ef52: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 04:49:23.940964 5274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:23.936549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38598 (local address 127.12.234.61:33601)
0809 04:49:23.936708 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:23.936713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:23.937145 (+ 432us) server_negotiation.cc:407] Connection header received
0809 04:49:23.937302 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:23.937306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:23.937369 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:23.937442 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:23.937804 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.938476 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.939622 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.939985 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.940093 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:23.940460 (+ 367us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:23.940542 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:23.940600 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:23.940715 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 04:49:23.940768 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:49:23.951256 5353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:23.946426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37758 (local address 127.12.234.60:39615)
0809 04:49:23.946660 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:23.946666 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:23.946854 (+ 188us) server_negotiation.cc:407] Connection header received
0809 04:49:23.946957 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:23.946961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:23.947107 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:23.947200 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:23.947478 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.948137 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.949225 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.949642 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.949741 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:23.950157 (+ 416us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:23.950676 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:23.950786 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:23.950861 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 04:49:23.950919 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260809 04:49:23.965080 5541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:23.957134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45322 (local address 127.12.234.62:39467)
0809 04:49:23.958015 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:23.958020 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:23.958037 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:49:23.958136 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:23.958142 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:23.958345 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:23.958517 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:23.959441 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.960490 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.962751 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.963225 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.963377 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:23.963863 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:23.964282 (+ 419us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:23.964411 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:23.964588 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 04:49:23.964665 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":622,"threads_started":1}
I20260809 04:49:23.965227 5274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:23.957116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38614 (local address 127.12.234.61:33601)
0809 04:49:23.957299 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:23.957320 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:23.958371 (+ 1051us) server_negotiation.cc:407] Connection header received
0809 04:49:23.958629 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:23.958635 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:23.958751 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:23.958843 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:23.959730 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.960714 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.963225 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.963879 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.964014 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:23.964461 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:23.964585 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:23.964689 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:23.964899 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 04:49:23.964970 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 04:49:23.966014 5353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:23.958427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37772 (local address 127.12.234.60:39615)
0809 04:49:23.958588 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:23.958594 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:23.959258 (+ 664us) server_negotiation.cc:407] Connection header received
0809 04:49:23.959547 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:23.959554 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:23.959630 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:23.959714 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:23.960337 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.961386 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.963534 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:23.964221 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:23.964384 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:23.965252 (+ 868us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:23.965379 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:23.965444 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:23.965685 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 04:49:23.965752 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:49:23.970367 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 5122
I20260809 04:49:23.971335 5187 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:49:24.075785 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 5122
I20260809 04:49:24.100777 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 5202
I20260809 04:49:24.101768 5265 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:49:24.202351 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 5202
I20260809 04:49:24.226238 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 5277
I20260809 04:49:24.227222 5345 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:49:24.326558 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 5277
2026-08-09T04:49:24Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4474 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 04:49:24 dist-test-slave-1jjb krb5kdc[5549](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 04:49:24 dist-test-slave-1jjb krb5kdc[5549](info): set up 2 sockets
Aug 09 04:49:24 dist-test-slave-1jjb krb5kdc[5549](info): commencing operation
krb5kdc: starting...
W20260809 04:49:25.453486 13224 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/krb5kdc/test-user.keytab.
Aug 09 04:49:25 dist-test-slave-1jjb krb5kdc[5549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T04:49:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T04:49:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.234.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
Entry for principal kudu/127.12.234.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab.
I20260809 04:49:25.739015 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:46823
--webserver_interface=127.12.234.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:38889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:46823,127.12.234.61:34883,127.12.234.60:40001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 04:49:26.051436 5563 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 04:49:26.051761 5563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:26.051872 5563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:26.051995 5563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:26.058725 5563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:26.058836 5563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:26.058944 5563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:26.059029 5563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:26.059116 5563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:26.067833 5563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:38889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:46823,127.12.234.61:34883,127.12.234.60:40001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.62.keytab
--principal=kudu/127.12.234.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.62:46823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.234.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:49:26.069033 5563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:26.070294 5563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:26.081948 5569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:26.082010 5573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:26.082046 5570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:26.082947 5563 server_base.cc:1034] running on GCE node
I20260809 04:49:26.083698 5563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:26.085023 5563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:26.086246 5563 hybrid_clock.cc:648] HybridClock initialized: now 1786250966086183 us; error 29 us; skew 500 ppm
I20260809 04:49:26.086587 5563 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:49:26 dist-test-slave-1jjb krb5kdc[5549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250966, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:26.089874 5563 init.cc:377] Logged in from keytab as kudu/127.12.234.62@KRBTEST.COM (short username kudu)
I20260809 04:49:26.091442 5563 webserver.cc:460] Webserver started at http://127.12.234.62:39915/ using document root <none> and password file <none>
I20260809 04:49:26.092221 5563 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:26.092360 5563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:26.092691 5563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:26.095569 5563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-0/data/instance:
uuid: "fedd0ec1c3f44297a60a05189d2ab645"
format_stamp: "Formatted at 2026-08-09 04:49:26 on dist-test-slave-1jjb"
server_key: "c7a71534b70aec0dbb7315a48f6ffc2e"
server_key_iv: "76819cf8ce2e57cdb08c08c798094a8a"
server_key_version: "encryptionkey@0"
I20260809 04:49:26.096403 5563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-0/wal/instance:
uuid: "fedd0ec1c3f44297a60a05189d2ab645"
format_stamp: "Formatted at 2026-08-09 04:49:26 on dist-test-slave-1jjb"
server_key: "c7a71534b70aec0dbb7315a48f6ffc2e"
server_key_iv: "76819cf8ce2e57cdb08c08c798094a8a"
server_key_version: "encryptionkey@0"
I20260809 04:49:26.101099 5563 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 04:49:26.104554 5580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:26.105911 5563 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 04:49:26.106106 5563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-0/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-0/wal
uuid: "fedd0ec1c3f44297a60a05189d2ab645"
format_stamp: "Formatted at 2026-08-09 04:49:26 on dist-test-slave-1jjb"
server_key: "c7a71534b70aec0dbb7315a48f6ffc2e"
server_key_iv: "76819cf8ce2e57cdb08c08c798094a8a"
server_key_version: "encryptionkey@0"
I20260809 04:49:26.106318 5563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:26.134212 5563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:26.138029 5563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:26.138302 5563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:26.157325 5563 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.62:46823
I20260809 04:49:26.157377 5632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.62:46823 every 8 connection(s)
I20260809 04:49:26.158605 5563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-0/data/info.pb
I20260809 04:49:26.162081 5633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:26.162858 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 5563
I20260809 04:49:26.163045 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-0/wal/instance
I20260809 04:49:26.163491 13224 external_mini_cluster.cc:1428] Setting key ed8d3f1e9d20c62791593f8ea545d604
I20260809 04:49:26.168560 5633 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46823 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:49:26 dist-test-slave-1jjb krb5kdc[5549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250966, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.234.61@KRBTEST.COM; defaulting to no policy
I20260809 04:49:26.198637 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.172017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46823 (local address 127.0.0.1:47506)
0809 04:49:26.180361 (+ 8344us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:26.180412 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:26.182340 (+ 1928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:26.183139 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:26.183147 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:26.183820 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:26.184221 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:26.184240 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:26.185759 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:26.185764 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:26.186803 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:26.186815 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:26.187732 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:26.187738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:26.187929 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:26.188701 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:26.188723 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:26.191165 (+ 2442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:26.194496 (+ 3331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:26.194509 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:26.194520 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:26.194889 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:26.195306 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:26.195311 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:26.195315 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:26.195478 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:26.196249 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:26.196257 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:26.196472 (+ 215us) client_negotiation.cc:769] Sending connection context
0809 04:49:26.196717 (+ 245us) client_negotiation.cc:240] Negotiation successful
0809 04:49:26.197339 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8090,"mutex_wait_us":176,"thread_start_us":7838,"threads_started":1}
I20260809 04:49:26.198696 5636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.172016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47506 (local address 127.12.234.62:46823)
0809 04:49:26.181065 (+ 9049us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:26.181075 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:26.181134 (+ 59us) server_negotiation.cc:407] Connection header received
0809 04:49:26.182698 (+ 1564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:26.182704 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:26.182917 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:26.183120 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:26.184442 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:26.185591 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:26.186952 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:26.187496 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:26.187701 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:26.191387 (+ 3686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:26.191401 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:26.191412 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:26.191439 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:26.194353 (+ 2914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:26.195043 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:26.195050 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:26.195052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:26.195195 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:26.195595 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:26.195600 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:26.195609 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:26.196048 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:26.196310 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:26.196858 (+ 548us) server_negotiation.cc:299] Negotiation successful
0809 04:49:26.197219 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":8834,"spinlock_wait_cycles":13568,"thread_start_us":8602,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.61@KRBTEST.COM" created.
I20260809 04:49:26.201714 5633 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34883 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:49:26.202862 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.202479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:34883 (local address 127.0.0.1:38896)
0809 04:49:26.202641 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:26.202756 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 04:49:26.203125 5583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34883: connect: Connection refused (error 111)
I20260809 04:49:26.204766 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.204420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:34883 (local address 127.0.0.1:38902)
0809 04:49:26.204588 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:26.204652 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 04:49:26.204999 5633 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.61:34883: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34883: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.61.keytab.
Entry for principal kudu/127.12.234.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.61.keytab.
I20260809 04:49:26.236646 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.61:34883
--webserver_interface=127.12.234.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.61.keytab
--principal=kudu/127.12.234.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:38889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:46823,127.12.234.61:34883,127.12.234.60:40001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:49:26.254344 5633 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34883 } attempt: 1
I20260809 04:49:26.255618 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.254987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:34883 (local address 127.0.0.1:38904)
0809 04:49:26.255251 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:26.255432 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260809 04:49:26.256755 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.256314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:34883 (local address 127.0.0.1:38918)
0809 04:49:26.256495 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:26.256587 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 04:49:26.256973 5633 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.61:34883: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34883: connect: Connection refused (error 111)
I20260809 04:49:26.325237 5633 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34883 } attempt: 2
I20260809 04:49:26.326380 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.325850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:34883 (local address 127.0.0.1:38930)
0809 04:49:26.326109 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:26.326214 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 04:49:26.327457 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.327057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:34883 (local address 127.0.0.1:38944)
0809 04:49:26.327201 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:26.327298 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 04:49:26.327644 5633 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.61:34883: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34883: connect: Connection refused (error 111)
I20260809 04:49:26.418944 5633 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34883 } attempt: 3
I20260809 04:49:26.420364 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.419674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:34883 (local address 127.0.0.1:38952)
0809 04:49:26.419977 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:26.420171 (+ 194us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260809 04:49:26.421542 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.421075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:34883 (local address 127.0.0.1:38964)
0809 04:49:26.421267 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:26.421363 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260809 04:49:26.421741 5633 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.61:34883: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34883: connect: Connection refused (error 111)
W20260809 04:49:26.556895 5640 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 04:49:26.557240 5640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:26.557332 5640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:26.557461 5640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:26.564711 5640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:26.564790 5640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:26.564901 5640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:26.564992 5640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:26.565096 5640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:26.564987 5633 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34883 } attempt: 4
I20260809 04:49:26.566236 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.565622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:34883 (local address 127.0.0.1:38966)
0809 04:49:26.565966 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:26.566069 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260809 04:49:26.567408 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.566968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:34883 (local address 127.0.0.1:38982)
0809 04:49:26.567157 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:26.567255 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34883: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260809 04:49:26.567682 5633 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.61:34883: Network error: Client connection negotiation failed: client connection to 127.12.234.61:34883: connect: Connection refused (error 111)
I20260809 04:49:26.575445 5640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:38889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:46823,127.12.234.61:34883,127.12.234.60:40001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.61.keytab
--principal=kudu/127.12.234.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.61:34883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.234.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:49:26.576785 5640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:26.578045 5640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:26.590184 5646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:26.590113 5647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:26.590160 5650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:26.591238 5640 server_base.cc:1034] running on GCE node
I20260809 04:49:26.592017 5640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:26.593442 5640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:26.594666 5640 hybrid_clock.cc:648] HybridClock initialized: now 1786250966594611 us; error 33 us; skew 500 ppm
I20260809 04:49:26.595006 5640 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:49:26 dist-test-slave-1jjb krb5kdc[5549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250966, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:26.598304 5640 init.cc:377] Logged in from keytab as kudu/127.12.234.61@KRBTEST.COM (short username kudu)
I20260809 04:49:26.599686 5640 webserver.cc:460] Webserver started at http://127.12.234.61:34925/ using document root <none> and password file <none>
I20260809 04:49:26.600414 5640 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:26.600540 5640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:26.600850 5640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:26.603658 5640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-1/data/instance:
uuid: "7d36f3f7c2ca4f89b79da8f915c02d23"
format_stamp: "Formatted at 2026-08-09 04:49:26 on dist-test-slave-1jjb"
server_key: "b1220759a0377e942ffced720463d299"
server_key_iv: "0cb36365945219528a686ec2282a39eb"
server_key_version: "encryptionkey@0"
I20260809 04:49:26.604470 5640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-1/wal/instance:
uuid: "7d36f3f7c2ca4f89b79da8f915c02d23"
format_stamp: "Formatted at 2026-08-09 04:49:26 on dist-test-slave-1jjb"
server_key: "b1220759a0377e942ffced720463d299"
server_key_iv: "0cb36365945219528a686ec2282a39eb"
server_key_version: "encryptionkey@0"
I20260809 04:49:26.609282 5640 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 04:49:26.612773 5658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:26.614131 5640 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:49:26.614326 5640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-1/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-1/wal
uuid: "7d36f3f7c2ca4f89b79da8f915c02d23"
format_stamp: "Formatted at 2026-08-09 04:49:26 on dist-test-slave-1jjb"
server_key: "b1220759a0377e942ffced720463d299"
server_key_iv: "0cb36365945219528a686ec2282a39eb"
server_key_version: "encryptionkey@0"
I20260809 04:49:26.614535 5640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:26.638620 5640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:26.642562 5640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:26.642822 5640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:26.660022 5640 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.61:34883
I20260809 04:49:26.660130 5714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.61:34883 every 8 connection(s)
I20260809 04:49:26.661161 5640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-1/data/info.pb
I20260809 04:49:26.664140 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 5640
I20260809 04:49:26.664319 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-1/wal/instance
I20260809 04:49:26.664765 13224 external_mini_cluster.cc:1428] Setting key 9b082d738a1d54be05d6c7582e49f8b3
I20260809 04:49:26.664921 5715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:26.671324 5715 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46823 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:49:26 dist-test-slave-1jjb krb5kdc[5549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250966, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.61@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:49:26.693107 5636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.673758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47512 (local address 127.12.234.62:46823)
0809 04:49:26.673962 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:26.673966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:26.675165 (+ 1199us) server_negotiation.cc:407] Connection header received
0809 04:49:26.677679 (+ 2514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:26.677683 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:26.677751 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:26.677839 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:26.679539 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:26.680623 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:26.681984 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:26.682742 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:26.682886 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:26.686279 (+ 3393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:26.686288 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:26.686295 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:26.686346 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:26.689970 (+ 3624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:26.690679 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:26.690687 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:26.690693 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:26.690832 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:26.691190 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:26.691197 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:26.691202 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:26.691621 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:26.691782 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:26.692377 (+ 595us) server_negotiation.cc:299] Negotiation successful
0809 04:49:26.692674 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 04:49:26.693585 5718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.674241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46823 (local address 127.0.0.1:47512)
0809 04:49:26.674976 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:26.675038 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:26.677418 (+ 2380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:26.678060 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:26.678070 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:26.678785 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:26.679326 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:26.679356 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:26.680770 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:26.680773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:26.681816 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:26.681827 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:26.682942 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:26.682946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:26.683135 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:26.683859 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:26.683879 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:26.686050 (+ 2171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:26.690169 (+ 4119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:26.690175 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:26.690184 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:26.690559 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:26.690931 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:26.690934 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:26.690936 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:26.691088 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:26.691744 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:26.691749 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:26.691988 (+ 239us) client_negotiation.cc:769] Sending connection context
0809 04:49:26.692245 (+ 257us) client_negotiation.cc:240] Negotiation successful
0809 04:49:26.692591 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":300,"threads_started":1}
I20260809 04:49:26.695335 5715 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34883 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.12.234.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.234.60@KRBTEST.COM" created.
Aug 09 04:49:26 dist-test-slave-1jjb krb5kdc[5549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250966, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.61@KRBTEST.COM for kudu/127.12.234.61@KRBTEST.COM
I20260809 04:49:26.709913 5718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.696168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:34883 (local address 127.0.0.1:38994)
0809 04:49:26.696324 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:26.696364 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:26.696478 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:26.697088 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:26.697092 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:26.697532 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:26.697657 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:26.697664 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:26.698938 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:26.698945 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:26.699680 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:26.699689 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:26.700461 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:26.700465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:26.700652 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:26.701165 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:26.701182 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:26.703221 (+ 2039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:26.707272 (+ 4051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:26.707276 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:26.707278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:26.707660 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:26.708162 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:26.708166 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:26.708169 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:26.708309 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:26.709031 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:26.709034 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:26.709173 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 04:49:26.709281 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 04:49:26.709555 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":4}
I20260809 04:49:26.710412 5720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.696249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38994 (local address 127.12.234.61:34883)
0809 04:49:26.696765 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:26.696769 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:26.696789 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:49:26.696823 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:26.696827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:26.696962 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:26.697098 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:26.697814 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:26.698777 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:26.699816 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:26.700342 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:26.700455 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:26.703453 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:26.703469 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:26.703486 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:26.703527 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:26.707092 (+ 3565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:26.707789 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:26.707798 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:26.707804 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:26.707950 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:26.708446 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:26.708453 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:26.708459 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:26.708909 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:26.709145 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:26.709483 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 04:49:26.709755 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":394,"thread_start_us":265,"threads_started":1}
I20260809 04:49:26.713063 5715 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:49:26.714043 5718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.713629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:40001 (local address 127.0.0.1:48320)
0809 04:49:26.713779 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:26.713907 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260809 04:49:26.714352 5661 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
I20260809 04:49:26.715853 5718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.715433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:40001 (local address 127.0.0.1:48324)
0809 04:49:26.715615 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:26.715715 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 04:49:26.716142 5715 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.60:40001: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.234.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.60.keytab.
Entry for principal kudu/127.12.234.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.60.keytab.
I20260809 04:49:26.736935 13224 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
/tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.60:40001
--webserver_interface=127.12.234.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.60.keytab
--principal=kudu/127.12.234.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.234.20:38889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.234.62:46823,127.12.234.61:34883,127.12.234.60:40001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 04:49:26.765499 5715 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } attempt: 1
I20260809 04:49:26.766731 5718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.766219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:40001 (local address 127.0.0.1:48326)
0809 04:49:26.766432 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:26.766552 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 04:49:26.767738 5718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.767327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:40001 (local address 127.0.0.1:48332)
0809 04:49:26.767513 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:26.767600 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 04:49:26.767921 5715 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.60:40001: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
I20260809 04:49:26.836165 5715 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } attempt: 2
I20260809 04:49:26.837383 5718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.836848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:40001 (local address 127.0.0.1:48334)
0809 04:49:26.837084 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:26.837194 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260809 04:49:26.838446 5718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.838080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:40001 (local address 127.0.0.1:48348)
0809 04:49:26.838233 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:26.838316 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 04:49:26.838680 5715 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.60:40001: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
I20260809 04:49:26.866926 5633 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34883 } attempt: 5
Aug 09 04:49:26 dist-test-slave-1jjb krb5kdc[5549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250966, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for kudu/127.12.234.61@KRBTEST.COM
I20260809 04:49:26.880509 5720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.867860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38996 (local address 127.12.234.61:34883)
0809 04:49:26.868026 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:26.868031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:26.868051 (+ 20us) server_negotiation.cc:407] Connection header received
0809 04:49:26.868158 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:26.868163 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:26.868233 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:26.868337 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:26.869136 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:26.869887 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:26.871033 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:26.871490 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:26.871622 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:26.875039 (+ 3417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:26.875044 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:26.875048 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:26.875077 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:26.877934 (+ 2857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:26.878768 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:26.878772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:26.878774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:26.878900 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:26.879273 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:26.879275 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:26.879277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:26.879521 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:26.879623 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:26.880055 (+ 432us) server_negotiation.cc:299] Negotiation successful
0809 04:49:26.880265 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 04:49:26.880620 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.867614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:34883 (local address 127.0.0.1:38996)
0809 04:49:26.867797 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:26.867831 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:26.867961 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:26.868403 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:26.868407 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:26.868787 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:26.868958 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:26.868966 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:26.870030 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:26.870036 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:26.870888 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:26.870899 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:26.871634 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:26.871640 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:26.871866 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:26.872571 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:26.872588 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:26.874839 (+ 2251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:26.878099 (+ 3260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:26.878106 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:26.878111 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:26.878652 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:26.879005 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:26.879009 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:26.879013 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:26.879169 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:26.879637 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:26.879641 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:26.879797 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 04:49:26.879970 (+ 173us) client_negotiation.cc:240] Negotiation successful
0809 04:49:26.880264 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 04:49:26.881738 5633 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } has no permanent_uuid. Determining permanent_uuid...
I20260809 04:49:26.882766 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.882279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:40001 (local address 127.0.0.1:48360)
0809 04:49:26.882489 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:26.882599 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 04:49:26.883883 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.883381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:40001 (local address 127.0.0.1:48366)
0809 04:49:26.883582 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:26.883696 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260809 04:49:26.884145 5633 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.60:40001: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
I20260809 04:49:26.929917 5715 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } attempt: 3
I20260809 04:49:26.931177 5718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.930621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:40001 (local address 127.0.0.1:48380)
0809 04:49:26.930863 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:26.930979 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 04:49:26.932343 5718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.931854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:40001 (local address 127.0.0.1:48394)
0809 04:49:26.932119 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:26.932208 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
W20260809 04:49:26.932701 5715 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.60:40001: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
I20260809 04:49:26.935395 5633 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } attempt: 1
I20260809 04:49:26.936415 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.935933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:40001 (local address 127.0.0.1:48398)
0809 04:49:26.936144 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:26.936236 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 04:49:26.937397 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:26.937036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:40001 (local address 127.0.0.1:48402)
0809 04:49:26.937185 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:26.937274 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 04:49:26.937608 5633 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.60:40001: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
I20260809 04:49:27.005852 5633 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } attempt: 2
I20260809 04:49:27.007125 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.006532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:40001 (local address 127.0.0.1:48406)
0809 04:49:27.006793 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:27.006922 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 04:49:27.008313 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.007889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:40001 (local address 127.0.0.1:48410)
0809 04:49:27.008045 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:27.008167 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 04:49:27.008586 5633 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.60:40001: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
W20260809 04:49:27.059782 5724 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 04:49:27.060071 5724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 04:49:27.060186 5724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 04:49:27.060297 5724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 04:49:27.066859 5724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 04:49:27.066928 5724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 04:49:27.067019 5724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 04:49:27.067085 5724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 04:49:27.067162 5724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 04:49:27.075937 5715 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } attempt: 4
I20260809 04:49:27.076360 5724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.234.20:38889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.234.62:46823,127.12.234.61:34883,127.12.234.60:40001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/krb5kdc/kudu_127.12.234.60.keytab
--principal=kudu/127.12.234.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.234.60:40001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.234.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 03:58:02 UTC on ccycloud.quasar-mitiho.root.comops.site
build id 4575794
ASAN enabled
I20260809 04:49:27.077107 5718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.076599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:40001 (local address 127.0.0.1:48422)
0809 04:49:27.076817 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:27.076929 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 04:49:27.077773 5724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 04:49:27.078125 5718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.077775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:40001 (local address 127.0.0.1:48426)
0809 04:49:27.077962 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:27.078025 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 04:49:27.078377 5715 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.60:40001: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
I20260809 04:49:27.079063 5724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 04:49:27.090063 5733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:27.090114 5731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 04:49:27.090296 5730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 04:49:27.090920 5724 server_base.cc:1034] running on GCE node
I20260809 04:49:27.091698 5724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 04:49:27.093087 5724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 04:49:27.094310 5724 hybrid_clock.cc:648] HybridClock initialized: now 1786250967094247 us; error 34 us; skew 500 ppm
I20260809 04:49:27.094638 5724 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 04:49:27 dist-test-slave-1jjb krb5kdc[5549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250967, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 04:49:27.097666 5724 init.cc:377] Logged in from keytab as kudu/127.12.234.60@KRBTEST.COM (short username kudu)
I20260809 04:49:27.098978 5724 webserver.cc:460] Webserver started at http://127.12.234.60:44863/ using document root <none> and password file <none>
I20260809 04:49:27.099649 5724 fs_manager.cc:359] Metadata directory not provided
I20260809 04:49:27.099762 5724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 04:49:27.100157 5724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 04:49:27.102941 5724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-2/data/instance:
uuid: "eb24fbf198644aa5bcee7a2e0b4f2b82"
format_stamp: "Formatted at 2026-08-09 04:49:27 on dist-test-slave-1jjb"
server_key: "29c56909f2c45395d9c168070e0bcfe4"
server_key_iv: "2b86766f160ea3829502cb98b2342023"
server_key_version: "encryptionkey@0"
I20260809 04:49:27.103719 5724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-2/wal/instance:
uuid: "eb24fbf198644aa5bcee7a2e0b4f2b82"
format_stamp: "Formatted at 2026-08-09 04:49:27 on dist-test-slave-1jjb"
server_key: "29c56909f2c45395d9c168070e0bcfe4"
server_key_iv: "2b86766f160ea3829502cb98b2342023"
server_key_version: "encryptionkey@0"
I20260809 04:49:27.108584 5724 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.004s
I20260809 04:49:27.111974 5742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 04:49:27.113452 5724 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 04:49:27.113633 5724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-2/data,/tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-2/wal
uuid: "eb24fbf198644aa5bcee7a2e0b4f2b82"
format_stamp: "Formatted at 2026-08-09 04:49:27 on dist-test-slave-1jjb"
server_key: "29c56909f2c45395d9c168070e0bcfe4"
server_key_iv: "2b86766f160ea3829502cb98b2342023"
server_key_version: "encryptionkey@0"
I20260809 04:49:27.113883 5724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 04:49:27.114845 5633 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } attempt: 3
I20260809 04:49:27.115883 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.115365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:40001 (local address 127.0.0.1:48430)
0809 04:49:27.115590 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:27.115687 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 04:49:27.116909 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.116583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:40001 (local address 127.0.0.1:48438)
0809 04:49:27.116749 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:27.116812 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260809 04:49:27.117141 5633 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.234.60:40001: Network error: Client connection negotiation failed: client connection to 127.12.234.60:40001: connect: Connection refused (error 111)
I20260809 04:49:27.126425 5724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 04:49:27.130421 5724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 04:49:27.130686 5724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 04:49:27.148116 5724 rpc_server.cc:307] RPC server started. Bound to: 127.12.234.60:40001
I20260809 04:49:27.148188 5798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.234.60:40001 every 8 connection(s)
I20260809 04:49:27.149394 5724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-2/data/info.pb
I20260809 04:49:27.152004 13224 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu as pid 5724
I20260809 04:49:27.152209 13224 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQ3A_6j/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786250603326224-13224-0/minicluster-data/master-2/wal/instance
I20260809 04:49:27.152760 13224 external_mini_cluster.cc:1428] Setting key 03ef4323d8ee79bff3eb422d2421e5ce
I20260809 04:49:27.153546 5799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 04:49:27.162326 5799 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46823 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:49:27 dist-test-slave-1jjb krb5kdc[5549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.60@KRBTEST.COM
Aug 09 04:49:27 dist-test-slave-1jjb krb5kdc[5549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250967, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.60@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
I20260809 04:49:27.177567 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.154507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48444 (local address 127.12.234.60:40001)
0809 04:49:27.155257 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.155267 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.155302 (+ 35us) server_negotiation.cc:407] Connection header received
0809 04:49:27.156897 (+ 1595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.156921 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.157417 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.157865 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:27.159482 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.161282 (+ 1800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.163007 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.163997 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.164771 (+ 774us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:27.169689 (+ 4918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:27.169701 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:27.169713 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:27.169755 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:27.173906 (+ 4151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:27.174528 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:27.174535 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:27.174542 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:27.174710 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:27.175069 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:27.175073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:27.175079 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:27.175684 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:27.176013 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:27.176404 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 04:49:27.176761 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":375,"threads_started":1}
I20260809 04:49:27.182646 5803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.165120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.62:46823 (local address 127.0.0.1:47526)
0809 04:49:27.167443 (+ 2323us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:27.167497 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:27.167707 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:27.168357 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:27.168365 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:27.169169 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:27.169542 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:27.169555 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.170896 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.170907 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:27.172640 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:27.172657 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.173659 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.173664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:27.173803 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:27.174335 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:27.174350 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:27.176554 (+ 2204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:27.179843 (+ 3289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:27.179856 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:27.179862 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:27.180397 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:27.180846 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:27.180850 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:27.180854 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:27.181033 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:27.181578 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:27.181588 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:27.181787 (+ 199us) client_negotiation.cc:769] Sending connection context
0809 04:49:27.181987 (+ 200us) client_negotiation.cc:240] Negotiation successful
0809 04:49:27.182297 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2061,"thread_start_us":1935,"threads_started":1}
I20260809 04:49:27.182699 5636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.165558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47526 (local address 127.12.234.62:46823)
0809 04:49:27.165775 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.165781 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.167577 (+ 1796us) server_negotiation.cc:407] Connection header received
0809 04:49:27.167866 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.167873 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.167963 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.168039 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:27.169711 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.170743 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.172799 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.173481 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.173619 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:27.176767 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:27.176772 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:27.176775 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:27.176802 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:27.179726 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:27.180558 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:27.180563 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:27.180567 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:27.180721 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:27.181176 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:27.181185 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:27.181189 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:27.181432 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:27.181557 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:27.182102 (+ 545us) server_negotiation.cc:299] Negotiation successful
0809 04:49:27.182242 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 04:49:27.183923 5799 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34883 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:49:27 dist-test-slave-1jjb krb5kdc[5549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.62@KRBTEST.COM
Aug 09 04:49:27 dist-test-slave-1jjb krb5kdc[5549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250967, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.60@KRBTEST.COM for kudu/127.12.234.61@KRBTEST.COM
I20260809 04:49:27.195729 5804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.180222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47542 (local address 127.12.234.62:46823)
0809 04:49:27.180798 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.180802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.180820 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:49:27.180854 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.180857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.180918 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.181006 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:27.182381 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.183562 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.185165 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.185630 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.185716 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:27.190263 (+ 4547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:27.190276 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:27.190279 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:27.190340 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:27.193235 (+ 2895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:27.193846 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:27.193852 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:27.193855 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:27.193979 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:27.194375 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:27.194380 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:27.194383 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:27.194627 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:27.194891 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:27.195179 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 04:49:27.195396 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":263,"threads_started":1}
I20260809 04:49:27.197707 5803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.184678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.61:34883 (local address 127.0.0.1:39002)
0809 04:49:27.184834 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:27.184851 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:27.184942 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:27.185288 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:27.185291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:27.185572 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:27.185723 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:27.185733 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.186828 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.186837 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:27.188346 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:27.188363 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.189025 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.189031 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:27.189286 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:27.189799 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:27.189813 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:27.192060 (+ 2247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:27.195034 (+ 2974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:27.195041 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:27.195044 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:27.195382 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:27.195952 (+ 570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:27.195957 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:27.195960 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:27.196153 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:27.196784 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:27.196790 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:27.196946 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 04:49:27.197101 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 04:49:27.197372 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 04:49:27.197845 5720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.184692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39002 (local address 127.12.234.61:34883)
0809 04:49:27.184887 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.184893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.184911 (+ 18us) server_negotiation.cc:407] Connection header received
0809 04:49:27.185095 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.185100 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.185176 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.185269 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:27.185877 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.186680 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.188493 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.188868 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.189003 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:27.192261 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:27.192266 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:27.192269 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:27.192297 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:27.194900 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:27.195666 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:27.195674 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:27.195677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:27.195840 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:27.196294 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:27.196298 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:27.196301 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:27.196657 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:27.196837 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:27.197224 (+ 387us) server_negotiation.cc:299] Negotiation successful
0809 04:49:27.197440 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 04:49:27.199059 5799 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 04:49:27 dist-test-slave-1jjb krb5kdc[5549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250967, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.60@KRBTEST.COM for kudu/127.12.234.60@KRBTEST.COM
Aug 09 04:49:27 dist-test-slave-1jjb krb5kdc[5549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.234.61@KRBTEST.COM
I20260809 04:49:27.213227 5803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.199647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:40001 (local address 127.0.0.1:48460)
0809 04:49:27.199909 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:27.199924 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:27.200013 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:27.200494 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:27.200498 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:27.200889 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:27.201025 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:27.201051 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.201935 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.201938 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:27.202720 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:27.202729 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.203491 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.203501 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:27.203830 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:27.204536 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:27.204558 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:27.206980 (+ 2422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:27.210894 (+ 3914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:27.210901 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:27.210905 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:27.211241 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:27.211604 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:27.211611 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:27.211616 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:27.211736 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:27.212220 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:27.212225 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:27.212363 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 04:49:27.212497 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 04:49:27.212798 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182}
I20260809 04:49:27.213400 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.199787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48460 (local address 127.12.234.60:40001)
0809 04:49:27.199911 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.199916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.199965 (+ 49us) server_negotiation.cc:407] Connection header received
0809 04:49:27.200198 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.200202 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.200265 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.200331 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:27.201181 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.201828 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.202867 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.203300 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.203420 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:27.207179 (+ 3759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:27.207185 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:27.207192 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:27.207232 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:27.210758 (+ 3526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:27.211375 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:27.211380 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:27.211383 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:27.211503 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:27.211830 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:27.211834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:27.211839 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:27.212128 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:27.212253 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:27.212589 (+ 336us) server_negotiation.cc:299] Negotiation successful
0809 04:49:27.212955 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":149,"server-negotiator.queue_time_us":32}
I20260809 04:49:27.214823 5720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.197594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39012 (local address 127.12.234.61:34883)
0809 04:49:27.198047 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.198053 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.198069 (+ 16us) server_negotiation.cc:407] Connection header received
0809 04:49:27.198112 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.198115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.198180 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.198290 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:27.200049 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.201143 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.203299 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.203857 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.203979 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:27.209358 (+ 5379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:27.209366 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:27.209372 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:27.209426 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:27.212254 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:27.212894 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:27.212899 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:27.212903 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:27.213033 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:27.213500 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:27.213504 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:27.213508 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:27.213861 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:27.214126 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:27.214295 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 04:49:27.214470 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":11722,"threads_started":1}
I20260809 04:49:27.218791 5799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb24fbf198644aa5bcee7a2e0b4f2b82: Bootstrap starting.
I20260809 04:49:27.219278 13224 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 04:49:27.222026 5799 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb24fbf198644aa5bcee7a2e0b4f2b82: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:27.223001 5799 log.cc:824] T 00000000000000000000000000000000 P eb24fbf198644aa5bcee7a2e0b4f2b82: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:27.221627 5608 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "fedd0ec1c3f44297a60a05189d2ab645"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47542
I20260809 04:49:27.225576 5799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb24fbf198644aa5bcee7a2e0b4f2b82: No bootstrap required, opened a new log
I20260809 04:49:27.231946 5799 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb24fbf198644aa5bcee7a2e0b4f2b82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedd0ec1c3f44297a60a05189d2ab645" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46823 } } peers { permanent_uuid: "7d36f3f7c2ca4f89b79da8f915c02d23" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34883 } } peers { permanent_uuid: "eb24fbf198644aa5bcee7a2e0b4f2b82" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } }
I20260809 04:49:27.232292 5799 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb24fbf198644aa5bcee7a2e0b4f2b82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb24fbf198644aa5bcee7a2e0b4f2b82, State: Initialized, Role: FOLLOWER
I20260809 04:49:27.232738 5799 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb24fbf198644aa5bcee7a2e0b4f2b82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedd0ec1c3f44297a60a05189d2ab645" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46823 } } peers { permanent_uuid: "7d36f3f7c2ca4f89b79da8f915c02d23" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34883 } } peers { permanent_uuid: "eb24fbf198644aa5bcee7a2e0b4f2b82" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } }
I20260809 04:49:27.233516 5808 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb24fbf198644aa5bcee7a2e0b4f2b82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedd0ec1c3f44297a60a05189d2ab645" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46823 } } peers { permanent_uuid: "7d36f3f7c2ca4f89b79da8f915c02d23" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34883 } } peers { permanent_uuid: "eb24fbf198644aa5bcee7a2e0b4f2b82" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } } }
I20260809 04:49:27.233850 5808 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb24fbf198644aa5bcee7a2e0b4f2b82 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:27.234671 5799 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb24fbf198644aa5bcee7a2e0b4f2b82 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 04:49:27.243139 5819 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eb24fbf198644aa5bcee7a2e0b4f2b82: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 04:49:27.243264 5819 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 04:49:27.241047 5724 master_runner.cc:186] Error getting master registration for 127.12.234.62:46823: OK, instance_id { permanent_uuid: "fedd0ec1c3f44297a60a05189d2ab645" instance_seqno: 1786250966151940 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 04:49:27.251839 5724 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 04:49:27.294302 5633 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } attempt: 4
Aug 09 04:49:27 dist-test-slave-1jjb krb5kdc[5549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250966, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.62@KRBTEST.COM for kudu/127.12.234.60@KRBTEST.COM
I20260809 04:49:27.309898 5635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.294948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:40001 (local address 127.0.0.1:48466)
0809 04:49:27.295147 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:27.295167 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:27.295311 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:27.295678 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:27.295683 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:27.296180 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:27.296384 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:27.296395 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.297779 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.297783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:27.299088 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:27.299101 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.299960 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.299965 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:27.300145 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:27.300798 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:27.300820 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:27.303289 (+ 2469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:27.307242 (+ 3953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:27.307250 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:27.307253 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:27.307719 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:27.308103 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:27.308109 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:27.308114 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:27.308269 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:27.308860 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:27.308867 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:27.309023 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 04:49:27.309179 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 04:49:27.309501 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 04:49:27.309911 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.295149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48466 (local address 127.12.234.60:40001)
0809 04:49:27.295357 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.295363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.295399 (+ 36us) server_negotiation.cc:407] Connection header received
0809 04:49:27.295458 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.295463 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.295540 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.295639 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:27.296527 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.297655 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.299213 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.299823 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.299941 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:27.303458 (+ 3517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:27.303466 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:27.303476 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:27.303519 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:27.307100 (+ 3581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:27.307835 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:27.307840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:27.307844 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:27.307977 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:27.308374 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:27.308379 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:27.308385 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:27.308728 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:27.308881 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:27.309358 (+ 477us) server_negotiation.cc:299] Negotiation successful
0809 04:49:27.309566 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 04:49:27.314391 5633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645: Bootstrap starting.
I20260809 04:49:27.317396 5633 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:27.318650 5633 log.cc:824] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:27.321360 5633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645: No bootstrap required, opened a new log
I20260809 04:49:27.328459 5633 raft_consensus.cc:348] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedd0ec1c3f44297a60a05189d2ab645" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46823 } } peers { permanent_uuid: "7d36f3f7c2ca4f89b79da8f915c02d23" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34883 } } peers { permanent_uuid: "eb24fbf198644aa5bcee7a2e0b4f2b82" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } }
I20260809 04:49:27.328869 5633 raft_consensus.cc:729] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fedd0ec1c3f44297a60a05189d2ab645, State: Initialized, Role: FOLLOWER
I20260809 04:49:27.329360 5633 consensus_queue.cc:260] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedd0ec1c3f44297a60a05189d2ab645" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46823 } } peers { permanent_uuid: "7d36f3f7c2ca4f89b79da8f915c02d23" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34883 } } peers { permanent_uuid: "eb24fbf198644aa5bcee7a2e0b4f2b82" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } }
I20260809 04:49:27.330263 5822 sys_catalog.cc:455] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedd0ec1c3f44297a60a05189d2ab645" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46823 } } peers { permanent_uuid: "7d36f3f7c2ca4f89b79da8f915c02d23" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34883 } } peers { permanent_uuid: "eb24fbf198644aa5bcee7a2e0b4f2b82" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } } }
I20260809 04:49:27.330696 5822 sys_catalog.cc:458] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:27.331369 5633 sys_catalog.cc:564] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:27.331492 5608 raft_consensus.cc:482] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 04:49:27.331698 5608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:27.335911 5608 raft_consensus.cc:504] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedd0ec1c3f44297a60a05189d2ab645" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46823 } } peers { permanent_uuid: "7d36f3f7c2ca4f89b79da8f915c02d23" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34883 } } peers { permanent_uuid: "eb24fbf198644aa5bcee7a2e0b4f2b82" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } }
I20260809 04:49:27.338286 5608 leader_election.cc:288] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645 [CANDIDATE]: Term 1 election: Requested vote from peers 7d36f3f7c2ca4f89b79da8f915c02d23 (127.12.234.61:34883), eb24fbf198644aa5bcee7a2e0b4f2b82 (127.12.234.60:40001)
I20260809 04:49:27.339371 5772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fedd0ec1c3f44297a60a05189d2ab645" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "eb24fbf198644aa5bcee7a2e0b4f2b82"
I20260809 04:49:27.339871 5772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb24fbf198644aa5bcee7a2e0b4f2b82 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:27.343247 5772 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eb24fbf198644aa5bcee7a2e0b4f2b82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fedd0ec1c3f44297a60a05189d2ab645 in term 1.
I20260809 04:49:27.343914 5583 leader_election.cc:302] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eb24fbf198644aa5bcee7a2e0b4f2b82, fedd0ec1c3f44297a60a05189d2ab645; no voters:
I20260809 04:49:27.344341 5822 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 04:49:27.345046 5822 raft_consensus.cc:686] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645 [term 1 LEADER]: Becoming Leader. State: Replica: fedd0ec1c3f44297a60a05189d2ab645, State: Running, Role: LEADER
I20260809 04:49:27.345772 5822 consensus_queue.cc:237] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedd0ec1c3f44297a60a05189d2ab645" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46823 } } peers { permanent_uuid: "7d36f3f7c2ca4f89b79da8f915c02d23" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34883 } } peers { permanent_uuid: "eb24fbf198644aa5bcee7a2e0b4f2b82" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } }
I20260809 04:49:27.353286 5829 sys_catalog.cc:455] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fedd0ec1c3f44297a60a05189d2ab645. Latest consensus state: current_term: 1 leader_uuid: "fedd0ec1c3f44297a60a05189d2ab645" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedd0ec1c3f44297a60a05189d2ab645" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46823 } } peers { permanent_uuid: "7d36f3f7c2ca4f89b79da8f915c02d23" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34883 } } peers { permanent_uuid: "eb24fbf198644aa5bcee7a2e0b4f2b82" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } } }
I20260809 04:49:27.353690 5829 sys_catalog.cc:458] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:27.367229 5835 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 04:49:27.377574 5715 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } attempt: 5
W20260809 04:49:27.382640 5749 tablet.cc:2369] T 00000000000000000000000000000000 P eb24fbf198644aa5bcee7a2e0b4f2b82: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:49:27.382871 5835 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260809 04:49:27.390863 5586 tablet.cc:2369] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 04:49:27.393656 5847 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 04:49:27.402048 5720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.370860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39020 (local address 127.12.234.61:34883)
0809 04:49:27.371110 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.371118 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.373645 (+ 2527us) server_negotiation.cc:407] Connection header received
0809 04:49:27.374567 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.374576 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.374675 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.374755 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:27.379355 (+ 4600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.380709 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.385281 (+ 4572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.386096 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.386202 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:27.395084 (+ 8882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:27.395090 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:27.395095 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:27.395127 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:27.398822 (+ 3695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:27.399593 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:27.399599 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:27.399602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:27.399746 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:27.400205 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:27.400210 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:27.400212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:27.400533 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:27.400765 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:27.401419 (+ 654us) server_negotiation.cc:299] Negotiation successful
0809 04:49:27.401635 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
Aug 09 04:49:27 dist-test-slave-1jjb krb5kdc[5549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786250966, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.234.61@KRBTEST.COM for kudu/127.12.234.60@KRBTEST.COM
I20260809 04:49:27.410795 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.373791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48472 (local address 127.12.234.60:40001)
0809 04:49:27.373981 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.373989 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.392631 (+ 18642us) server_negotiation.cc:407] Connection header received
0809 04:49:27.392903 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.392909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.392994 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.393072 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:27.395089 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.396244 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.398074 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.398653 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.398742 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:27.401907 (+ 3165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:27.401916 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:27.401920 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:27.401958 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:27.407241 (+ 5283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:27.407959 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:27.407965 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:27.407968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:27.408182 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:27.408635 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:27.408639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:27.408642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:27.408924 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:27.409178 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:27.410206 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0809 04:49:27.410437 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 04:49:27.398023 5563 master_runner.cc:186] Error getting master registration for 127.12.234.61:34883: OK, instance_id { permanent_uuid: "7d36f3f7c2ca4f89b79da8f915c02d23" instance_seqno: 1786250966655304 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 04:49:27.413039 5563 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 04:49:27.418110 5804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.375067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47550 (local address 127.12.234.62:46823)
0809 04:49:27.375266 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.375271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.397722 (+ 22451us) server_negotiation.cc:407] Connection header received
0809 04:49:27.400227 (+ 2505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.400232 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.400359 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.400427 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:27.404004 (+ 3577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.405037 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.406800 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.407498 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.407587 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:27.410954 (+ 3367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:27.410963 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:27.410970 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:27.411010 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:27.414766 (+ 3756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:27.415555 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:27.415560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:27.415563 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:27.415707 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:27.416215 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:27.416220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:27.416223 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:27.416556 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:27.416823 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:27.417471 (+ 648us) server_negotiation.cc:299] Negotiation successful
0809 04:49:27.417748 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 04:49:27.420869 5718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.378129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.234.60:40001 (local address 127.0.0.1:48486)
0809 04:49:27.378517 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0809 04:49:27.378536 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 04:49:27.378637 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 04:49:27.394532 (+ 15895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 04:49:27.394537 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 04:49:27.395025 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0809 04:49:27.395240 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:27.395254 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.397813 (+ 2559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.397818 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:27.399135 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 04:49:27.399149 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.401614 (+ 2465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.401619 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 04:49:27.401862 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:27.402915 (+ 1053us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 04:49:27.402942 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 04:49:27.405509 (+ 2567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 04:49:27.416245 (+ 10736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:27.416249 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:27.416252 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:27.416763 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:27.417527 (+ 764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 04:49:27.417531 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 04:49:27.417535 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 04:49:27.417726 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 04:49:27.419555 (+ 1829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 04:49:27.419559 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 04:49:27.419700 (+ 141us) client_negotiation.cc:769] Sending connection context
0809 04:49:27.419834 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 04:49:27.420462 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261}
I20260809 04:49:27.422026 5846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.378831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48486 (local address 127.12.234.60:40001)
0809 04:49:27.394157 (+ 15326us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.394163 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.394184 (+ 21us) server_negotiation.cc:407] Connection header received
0809 04:49:27.394228 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.394232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.394298 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.395403 (+ 1105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:27.395718 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.397050 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.399306 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.399906 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.402785 (+ 2879us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:27.407537 (+ 4752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 04:49:27.407546 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 04:49:27.407579 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 04:49:27.407614 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 04:49:27.416050 (+ 8436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:27.416986 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:27.416991 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:27.416994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:27.417325 (+ 331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 04:49:27.417934 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 04:49:27.417995 (+ 61us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 04:49:27.417999 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 04:49:27.419398 (+ 1399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 04:49:27.421257 (+ 1859us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:27.421404 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 04:49:27.421610 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15209,"thread_start_us":14983,"threads_started":1}
I20260809 04:49:27.427275 5715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d36f3f7c2ca4f89b79da8f915c02d23: Bootstrap starting.
I20260809 04:49:27.431318 5715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d36f3f7c2ca4f89b79da8f915c02d23: Neither blocks nor log segments found. Creating new log.
I20260809 04:49:27.432785 5715 log.cc:824] T 00000000000000000000000000000000 P 7d36f3f7c2ca4f89b79da8f915c02d23: Log is configured to *not* fsync() on all Append() calls
I20260809 04:49:27.434579 5772 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eb24fbf198644aa5bcee7a2e0b4f2b82 [term 1 FOLLOWER]: Refusing update from remote peer fedd0ec1c3f44297a60a05189d2ab645: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:49:27.437336 5829 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb24fbf198644aa5bcee7a2e0b4f2b82" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:49:27.446862 5715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d36f3f7c2ca4f89b79da8f915c02d23: No bootstrap required, opened a new log
I20260809 04:49:27.458196 5715 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d36f3f7c2ca4f89b79da8f915c02d23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedd0ec1c3f44297a60a05189d2ab645" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46823 } } peers { permanent_uuid: "7d36f3f7c2ca4f89b79da8f915c02d23" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34883 } } peers { permanent_uuid: "eb24fbf198644aa5bcee7a2e0b4f2b82" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } }
I20260809 04:49:27.458588 5715 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d36f3f7c2ca4f89b79da8f915c02d23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d36f3f7c2ca4f89b79da8f915c02d23, State: Initialized, Role: FOLLOWER
I20260809 04:49:27.459070 5715 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d36f3f7c2ca4f89b79da8f915c02d23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedd0ec1c3f44297a60a05189d2ab645" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46823 } } peers { permanent_uuid: "7d36f3f7c2ca4f89b79da8f915c02d23" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34883 } } peers { permanent_uuid: "eb24fbf198644aa5bcee7a2e0b4f2b82" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } }
I20260809 04:49:27.461997 5808 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb24fbf198644aa5bcee7a2e0b4f2b82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fedd0ec1c3f44297a60a05189d2ab645. Latest consensus state: current_term: 1 leader_uuid: "fedd0ec1c3f44297a60a05189d2ab645" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedd0ec1c3f44297a60a05189d2ab645" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46823 } } peers { permanent_uuid: "7d36f3f7c2ca4f89b79da8f915c02d23" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34883 } } peers { permanent_uuid: "eb24fbf198644aa5bcee7a2e0b4f2b82" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } } }
I20260809 04:49:27.462327 5808 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb24fbf198644aa5bcee7a2e0b4f2b82 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:27.463301 5854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d36f3f7c2ca4f89b79da8f915c02d23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedd0ec1c3f44297a60a05189d2ab645" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46823 } } peers { permanent_uuid: "7d36f3f7c2ca4f89b79da8f915c02d23" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34883 } } peers { permanent_uuid: "eb24fbf198644aa5bcee7a2e0b4f2b82" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } } }
I20260809 04:49:27.464886 5848 mvcc.cc:205] Tried to move back new op lower bound from 7316483962556813312 to 7316483962254090240. Current Snapshot: MvccSnapshot[applied={T|T < 7316483962556813312}]
I20260809 04:49:27.466538 5854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d36f3f7c2ca4f89b79da8f915c02d23 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:27.465744 5687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d36f3f7c2ca4f89b79da8f915c02d23 [term 0 FOLLOWER]: Advancing to term 1
I20260809 04:49:27.467067 5829 sys_catalog.cc:455] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fedd0ec1c3f44297a60a05189d2ab645" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedd0ec1c3f44297a60a05189d2ab645" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46823 } } peers { permanent_uuid: "7d36f3f7c2ca4f89b79da8f915c02d23" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34883 } } peers { permanent_uuid: "eb24fbf198644aa5bcee7a2e0b4f2b82" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } } }
I20260809 04:49:27.467341 5829 sys_catalog.cc:458] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:27.465937 5689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fedd0ec1c3f44297a60a05189d2ab645" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7d36f3f7c2ca4f89b79da8f915c02d23"
I20260809 04:49:27.469666 5851 mvcc.cc:205] Tried to move back new op lower bound from 7316483962556813312 to 7316483962254090240. Current Snapshot: MvccSnapshot[applied={T|T < 7316483962556813312}]
I20260809 04:49:27.480178 5687 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7d36f3f7c2ca4f89b79da8f915c02d23 [term 1 FOLLOWER]: Refusing update from remote peer fedd0ec1c3f44297a60a05189d2ab645: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 04:49:27.483520 5689 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7d36f3f7c2ca4f89b79da8f915c02d23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fedd0ec1c3f44297a60a05189d2ab645 in term 1.
I20260809 04:49:27.485756 5822 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d36f3f7c2ca4f89b79da8f915c02d23" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 04:49:27.465310 5715 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d36f3f7c2ca4f89b79da8f915c02d23 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 04:49:27.500859 5835 catalog_manager.cc:1347] Generated new cluster ID: ec0166ed453141279cef559cc46ac35b
I20260809 04:49:27.500952 5835 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 04:49:27.523896 5808 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb24fbf198644aa5bcee7a2e0b4f2b82 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fedd0ec1c3f44297a60a05189d2ab645" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedd0ec1c3f44297a60a05189d2ab645" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46823 } } peers { permanent_uuid: "7d36f3f7c2ca4f89b79da8f915c02d23" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34883 } } peers { permanent_uuid: "eb24fbf198644aa5bcee7a2e0b4f2b82" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } } }
I20260809 04:49:27.524194 5808 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb24fbf198644aa5bcee7a2e0b4f2b82 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:27.539459 5835 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 04:49:27.541066 5835 catalog_manager.cc:1504] Loading token signing keys...
I20260809 04:49:27.556756 5857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d36f3f7c2ca4f89b79da8f915c02d23 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fedd0ec1c3f44297a60a05189d2ab645" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedd0ec1c3f44297a60a05189d2ab645" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46823 } } peers { permanent_uuid: "7d36f3f7c2ca4f89b79da8f915c02d23" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34883 } } peers { permanent_uuid: "eb24fbf198644aa5bcee7a2e0b4f2b82" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } } }
I20260809 04:49:27.557159 5857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d36f3f7c2ca4f89b79da8f915c02d23 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:27.557498 5854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d36f3f7c2ca4f89b79da8f915c02d23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fedd0ec1c3f44297a60a05189d2ab645. Latest consensus state: current_term: 1 leader_uuid: "fedd0ec1c3f44297a60a05189d2ab645" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedd0ec1c3f44297a60a05189d2ab645" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46823 } } peers { permanent_uuid: "7d36f3f7c2ca4f89b79da8f915c02d23" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34883 } } peers { permanent_uuid: "eb24fbf198644aa5bcee7a2e0b4f2b82" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } } }
I20260809 04:49:27.557729 5854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d36f3f7c2ca4f89b79da8f915c02d23 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 04:49:27.559171 5822 sys_catalog.cc:455] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fedd0ec1c3f44297a60a05189d2ab645" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fedd0ec1c3f44297a60a05189d2ab645" member_type: VOTER last_known_addr { host: "127.12.234.62" port: 46823 } } peers { permanent_uuid: "7d36f3f7c2ca4f89b79da8f915c02d23" member_type: VOTER last_known_addr { host: "127.12.234.61" port: 34883 } } peers { permanent_uuid: "eb24fbf198644aa5bcee7a2e0b4f2b82" member_type: VOTER last_known_addr { host: "127.12.234.60" port: 40001 } } }
I20260809 04:49:27.559441 5822 sys_catalog.cc:458] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645 [sys.catalog]: This master's current role is: LEADER
I20260809 04:49:27.568084 5835 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fedd0ec1c3f44297a60a05189d2ab645: Generated new TSK 0
I20260809 04:49:27.568960 5835 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 04:49:27.594332 5874 catalog_manager.cc:1259] Loaded cluster ID: ec0166ed453141279cef559cc46ac35b
I20260809 04:49:27.594501 5874 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7d36f3f7c2ca4f89b79da8f915c02d23: loading cluster ID for follower catalog manager: success
I20260809 04:49:27.598035 5874 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7d36f3f7c2ca4f89b79da8f915c02d23: acquiring CA information for follower catalog manager: success
I20260809 04:49:27.600299 5874 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7d36f3f7c2ca4f89b79da8f915c02d23: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 04:49:27.706084 5720 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:27.691780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39026 (local address 127.12.234.61:34883)
0809 04:49:27.692002 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.692009 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.703279 (+ 11270us) server_negotiation.cc:407] Connection header received
0809 04:49:27.704257 (+ 978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.704262 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.704342 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.704428 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:27.705877 (+ 1449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39026: BlockingRecv error: recv got EOF from 127.0.0.1:39026 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260809 04:49:27.706840 5804 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:27.695326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47558 (local address 127.12.234.62:46823)
0809 04:49:27.695778 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.695784 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.704549 (+ 8765us) server_negotiation.cc:407] Connection header received
0809 04:49:27.705316 (+ 767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.705323 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.705412 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.705488 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:27.706584 (+ 1096us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47558: BlockingRecv error: recv got EOF from 127.0.0.1:47558 (error 108)
Metrics: {"server-negotiator.queue_time_us":279}
W20260809 04:49:27.709038 5846 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:27.705042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48502 (local address 127.12.234.60:40001)
0809 04:49:27.705231 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.705237 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.706843 (+ 1606us) server_negotiation.cc:407] Connection header received
0809 04:49:27.707629 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.707635 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.707729 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.707795 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:27.708833 (+ 1038us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48502: BlockingRecv error: recv got EOF from 127.0.0.1:48502 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 04:49:27.760274 5720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.745340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39036 (local address 127.12.234.61:34883)
0809 04:49:27.745525 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.745531 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.752869 (+ 7338us) server_negotiation.cc:407] Connection header received
0809 04:49:27.753367 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.753374 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.753595 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.753740 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:27.754460 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.755470 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.757379 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.758014 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.758103 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:27.758789 (+ 686us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:27.759483 (+ 694us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:27.759592 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:27.759820 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 04:49:27.759925 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260809 04:49:27.796582 5846 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:27.794263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48506 (local address 127.12.234.60:40001)
0809 04:49:27.794438 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.794444 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.794966 (+ 522us) server_negotiation.cc:407] Connection header received
0809 04:49:27.795203 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.795209 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.795338 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.795442 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:27.796288 (+ 846us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48506: BlockingRecv error: recv got EOF from 127.0.0.1:48506 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 04:49:27.811033 5720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.800820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39050 (local address 127.12.234.61:34883)
0809 04:49:27.801045 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.801051 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.804586 (+ 3535us) server_negotiation.cc:407] Connection header received
0809 04:49:27.804754 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.804759 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.804848 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.804941 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:27.805373 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.806519 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.808433 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.809129 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.809256 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:27.809856 (+ 600us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:27.810003 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:27.810098 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:27.810358 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 04:49:27.810440 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260809 04:49:27.829984 5846 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:27.826248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48516 (local address 127.12.234.60:40001)
0809 04:49:27.826428 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.826433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.828564 (+ 2131us) server_negotiation.cc:407] Connection header received
0809 04:49:27.828736 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.828741 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.828826 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.828924 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:27.829776 (+ 852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48516: BlockingRecv error: recv got EOF from 127.0.0.1:48516 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 04:49:27.841522 5720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.835060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39054 (local address 127.12.234.61:34883)
0809 04:49:27.835215 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.835220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.835708 (+ 488us) server_negotiation.cc:407] Connection header received
0809 04:49:27.835841 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.835846 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.835920 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.836007 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:27.836459 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.837577 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.839512 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.840147 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.840278 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:27.840734 (+ 456us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:27.840874 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:27.840958 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:27.841101 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 04:49:27.841193 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 04:49:27.861927 5846 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:27.849614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48524 (local address 127.12.234.60:40001)
0809 04:49:27.849784 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.849790 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.860546 (+ 10756us) server_negotiation.cc:407] Connection header received
0809 04:49:27.860666 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.860672 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.860756 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.860857 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:27.861714 (+ 857us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48524: BlockingRecv error: recv got EOF from 127.0.0.1:48524 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 04:49:27.875671 5720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.869133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39060 (local address 127.12.234.61:34883)
0809 04:49:27.869351 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.869357 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.869879 (+ 522us) server_negotiation.cc:407] Connection header received
0809 04:49:27.870039 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.870044 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.870135 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.870226 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:27.870712 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.871708 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.873691 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.874361 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.874485 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:27.874938 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:27.875062 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:27.875148 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:27.875336 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 04:49:27.875411 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
W20260809 04:49:27.888322 5846 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:27.882166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48528 (local address 127.12.234.60:40001)
0809 04:49:27.882442 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.882447 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.887011 (+ 4564us) server_negotiation.cc:407] Connection header received
0809 04:49:27.887145 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.887150 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.887233 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.887332 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:27.888116 (+ 784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48528: BlockingRecv error: recv got EOF from 127.0.0.1:48528 (error 108)
Metrics: {"server-negotiator.queue_time_us":160}
I20260809 04:49:27.908025 5720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.901427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39062 (local address 127.12.234.61:34883)
0809 04:49:27.901582 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.901587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.902045 (+ 458us) server_negotiation.cc:407] Connection header received
0809 04:49:27.902197 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.902204 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.902290 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.902386 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:27.902813 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.904054 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.905842 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.906544 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.906677 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:27.907192 (+ 515us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:27.907340 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:27.907467 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:27.907656 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 04:49:27.907730 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 04:49:27.917917 5846 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:27.914534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48530 (local address 127.12.234.60:40001)
0809 04:49:27.914698 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.914703 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.916551 (+ 1848us) server_negotiation.cc:407] Connection header received
0809 04:49:27.916695 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.916700 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.916782 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.916893 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:27.917714 (+ 821us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48530: BlockingRecv error: recv got EOF from 127.0.0.1:48530 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 04:49:27.943395 5720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.936848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39070 (local address 127.12.234.61:34883)
0809 04:49:27.937008 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.937014 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.937564 (+ 550us) server_negotiation.cc:407] Connection header received
0809 04:49:27.937794 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.937799 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.937878 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.937971 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:27.938451 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.939527 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.941255 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.941856 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.942010 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:27.942493 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:27.942676 (+ 183us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:27.942818 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:27.943021 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 04:49:27.943107 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 04:49:27.952868 5846 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:27.950288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48532 (local address 127.12.234.60:40001)
0809 04:49:27.950523 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.950529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.951064 (+ 535us) server_negotiation.cc:407] Connection header received
0809 04:49:27.951349 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.951355 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.951449 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.951520 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:27.952637 (+ 1117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48532: BlockingRecv error: recv got EOF from 127.0.0.1:48532 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
I20260809 04:49:27.999773 5720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:27.993176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39084 (local address 127.12.234.61:34883)
0809 04:49:27.993403 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:27.993410 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:27.993991 (+ 581us) server_negotiation.cc:407] Connection header received
0809 04:49:27.994217 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:27.994223 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:27.994298 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:27.994404 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:27.994875 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.995924 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.997818 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:27.998390 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:27.998515 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:27.998976 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:27.999087 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:27.999169 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:27.999342 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 04:49:27.999429 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 04:49:28.012223 5846 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:28.009982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48544 (local address 127.12.234.60:40001)
0809 04:49:28.010183 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:28.010190 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:28.010687 (+ 497us) server_negotiation.cc:407] Connection header received
0809 04:49:28.010890 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:28.010908 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:28.010981 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:28.011064 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:28.011922 (+ 858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48544: BlockingRecv error: recv got EOF from 127.0.0.1:48544 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:49:28.084517 5720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:28.079309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39090 (local address 127.12.234.61:34883)
0809 04:49:28.079509 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:28.079517 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:28.079923 (+ 406us) server_negotiation.cc:407] Connection header received
0809 04:49:28.080211 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:28.080215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:28.080284 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:28.080388 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:28.080839 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:28.081543 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:28.082890 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:28.083280 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:28.083419 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:28.083813 (+ 394us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:28.083940 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:28.084034 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:28.084193 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 04:49:28.084270 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 04:49:28.092559 5846 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:28.090533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48548 (local address 127.12.234.60:40001)
0809 04:49:28.090708 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:28.090715 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:28.091125 (+ 410us) server_negotiation.cc:407] Connection header received
0809 04:49:28.091330 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:28.091334 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:28.091402 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:28.091518 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:28.092331 (+ 813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48548: BlockingRecv error: recv got EOF from 127.0.0.1:48548 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 04:49:28.229425 5720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:28.224236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39106 (local address 127.12.234.61:34883)
0809 04:49:28.224495 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:28.224504 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:28.224898 (+ 394us) server_negotiation.cc:407] Connection header received
0809 04:49:28.225126 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:28.225130 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:28.225198 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:28.225298 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:28.225723 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:28.226404 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:28.227718 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:28.228120 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:28.228261 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:28.228635 (+ 374us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:28.228745 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:28.228852 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:28.229030 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 04:49:28.229139 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 04:49:28.236443 5846 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 04:49:28.234692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48556 (local address 127.12.234.60:40001)
0809 04:49:28.234932 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:28.234938 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:28.235088 (+ 150us) server_negotiation.cc:407] Connection header received
0809 04:49:28.235278 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:28.235282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:28.235349 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:28.235432 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 04:49:28.236207 (+ 775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48556: BlockingRecv error: recv got EOF from 127.0.0.1:48556 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 04:49:28.245152 5819 catalog_manager.cc:1259] Loaded cluster ID: ec0166ed453141279cef559cc46ac35b
I20260809 04:49:28.245226 5819 catalog_manager.cc:1552] T 00000000000000000000000000000000 P eb24fbf198644aa5bcee7a2e0b4f2b82: loading cluster ID for follower catalog manager: success
I20260809 04:49:28.247597 5819 catalog_manager.cc:1574] T 00000000000000000000000000000000 P eb24fbf198644aa5bcee7a2e0b4f2b82: acquiring CA information for follower catalog manager: success
I20260809 04:49:28.249440 5819 catalog_manager.cc:1602] T 00000000000000000000000000000000 P eb24fbf198644aa5bcee7a2e0b4f2b82: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 04:49:28.501045 5720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:28.496255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39122 (local address 127.12.234.61:34883)
0809 04:49:28.496498 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:28.496504 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:28.496673 (+ 169us) server_negotiation.cc:407] Connection header received
0809 04:49:28.496890 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:28.496894 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:28.496955 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:28.497044 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:28.497417 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:28.498153 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:28.499459 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:28.499872 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:28.499995 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:28.500419 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:28.500527 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:28.500621 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:28.500749 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 04:49:28.500822 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260809 04:49:28.512025 5846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:28.506615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48566 (local address 127.12.234.60:40001)
0809 04:49:28.506833 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:28.506839 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:28.506962 (+ 123us) server_negotiation.cc:407] Connection header received
0809 04:49:28.507199 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:28.507203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:28.507404 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:28.507521 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:28.507931 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:28.508620 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:28.510038 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:28.510428 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:28.510517 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:28.510957 (+ 440us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:28.511479 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:28.511583 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:28.511695 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 04:49:28.511756 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 04:49:28.526224 5720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:28.517143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39130 (local address 127.12.234.61:34883)
0809 04:49:28.517296 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:28.517303 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:28.518006 (+ 703us) server_negotiation.cc:407] Connection header received
0809 04:49:28.518393 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:28.518399 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:28.518510 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:28.518579 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:28.519470 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:28.520698 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:28.523345 (+ 2647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:28.524257 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:28.524382 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:28.525203 (+ 821us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:28.525393 (+ 190us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:28.525459 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:28.525726 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 04:49:28.525827 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 04:49:28.531944 5846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:28.517997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48572 (local address 127.12.234.60:40001)
0809 04:49:28.518193 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:28.518200 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:28.526124 (+ 7924us) server_negotiation.cc:407] Connection header received
0809 04:49:28.526494 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:28.526502 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:28.526612 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:28.526682 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:28.527297 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:28.528301 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:28.530215 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:28.530819 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:28.530937 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:28.531267 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:28.531380 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:28.531465 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:28.531592 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 04:49:28.531663 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 04:49:28.534003 5998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 04:49:28.517177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47568 (local address 127.12.234.62:46823)
0809 04:49:28.527935 (+ 10758us) server_negotiation.cc:206] Beginning negotiation
0809 04:49:28.527941 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 04:49:28.527958 (+ 17us) server_negotiation.cc:407] Connection header received
0809 04:49:28.528076 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 04:49:28.528080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 04:49:28.528369 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 04:49:28.528540 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 04:49:28.529013 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:28.530113 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:28.531914 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 04:49:28.532435 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 04:49:28.532522 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 04:49:28.533176 (+ 654us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 04:49:28.533502 (+ 326us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 04:49:28.533585 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 04:49:28.533707 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 04:49:28.533771 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10627,"thread_start_us":10940,"threads_started":1}
I20260809 04:49:28.539534 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 5563
I20260809 04:49:28.540462 5628 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:49:28.644114 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 5563
I20260809 04:49:28.669255 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 5640
I20260809 04:49:28.670382 5710 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:49:28.770459 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 5640
I20260809 04:49:28.794036 13224 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu pid 5724
I20260809 04:49:28.795110 5794 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 04:49:28.896673 13224 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQ3A_6j/build/asan/bin/kudu with pid 5724
2026-08-09T04:49:28Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4569 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9044 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (365452 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST